Publications

Empirical Inference Autonomous Learning Conference Paper On the Transfer of Object-Centric Representation Learning Didolkar, A. R., Zadaianchuk, A., Goyal, A., Mozer, M. C., Bengio, Y., Martius*, G., Seitzer*, M. The Thirteenth International Conference on Learning Representations (ICLR), April 2025, *equal contribution (Published) URL BibTeX

Empirical Inference Conference Paper Preference Elicitation for Offline Reinforcement Learning Pace, A., Schölkopf, B., Rätsch, G., Ramponi, G. The Thirteenth International Conference on Learning Representations (ICLR), April 2025 (Published) arXiv BibTeX

Empirical Inference Conference Paper Standardizing Structural Causal Models Ormaniec*, W., Sussex*, S., Lorch*, L., Schölkopf, B., Krause, A. The Thirteenth International Conference on Learning Representations (ICLR), April 2025, *equal contribution (Published) arXiv BibTeX

Empirical Inference Conference Paper The Directionality of Optimization Trajectories in Neural Networks Singh, S. P., He, B., Hofmann, T., Schölkopf, B. The Thirteenth International Conference on Learning Representations (ICLR), April 2025 (Published) URL BibTeX

Empirical Inference Article The Fiction Machine Bottou, L., Schölkopf, B. SIAM News, 58(3), April 2025 (Published) URL BibTeX

Empirical Inference Conference Paper What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis Ormaniec, W., Dangel, F., Singh, S. P. The Thirteenth International Conference on Learning Representations (ICLR), April 2025 (Published) arXiv BibTeX

Empirical Inference Conference Paper Why AI Is WEIRD and Should Not Be This Way: Towards AI For Everyone, With Everyone, By Everyone Mihalcea*, R., Ignat*, O., Bai, L., Borah, A., Chiruzzo, L., Jin, Z., Kwizera, C., Nwatu, J., Poria, S., Solorio, T. The Thirty-Nineth AAAI Conference on Artificial Intelligence, AAAI 2025 (Senior Member Presentation Track), (27)28657-28670, (Editors: Toby Walsh, Julie Shah, Zico Kolter ), AAAI Press, April 2025, *equal contribution (Published)

Abstract ›

This paper presents a vision for creating AI systems that are inclusive at every stage of development, from data collection to model design and evaluation. We address key limitations in the current AI pipeline and its WEIRD* representation, such as lack of data diversity, biases in model performance, and narrow evaluation metrics. We also focus on the need for diverse representation among the developers of these systems, as well as incentives that are not skewed toward certain groups. We highlight opportunities to develop AI systems that are for everyone (with diverse stakeholders in mind), with everyone (inclusive of diverse data and annotators), and by everyone (designed and developed by a globally diverse workforce). *WEIRD = an acronym coined by Joseph Henrich to highlight the coverage limitations of many psychological studies, referring to populations that are Western, Educated, Industrialized, Rich, and Democratic; while we do not fully adopt this term for AI, as its current scope does not perfectly align with the WEIRD dimensions, we believe that today’s AI has a similarly "weird" coverage, particularly in terms of who is involved in its development and who benefits from it.

arXiv DOI URL BibTeX

Empirical Inference Conference Paper MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs Opedal*, A., Shirakami*, H., Schölkopf, B., Saparov, A., Sachan, M. The Thirteenth International Conference on Learning Representations (ICLR), April 2025, *equal contribution (Published) arXiv BibTeX

Empirical Inference Article Early warning of complex climate risk with integrated artificial intelligence Reichstein, M., Benson, V., Blunk, J., Camps-Valls, G., Creutzig, F., Fearnley, C. J., Han, B., Kornhuber, K., Rahaman, N., Schölkopf, B., Tárraga, J. M., Vinuesa, R., Dall, K., Denzler, J., Frank, D., Martini, G., Nganga, N., Maddix, D. C., Weldemariam, K. Nature Communications, 16(1), March 2025 (Published) DOI BibTeX

Empirical Inference Ph.D. Thesis Learning to Generalize Across Distribution Shifts Träuble, F. J. University of Tübingen, Germany, March 2025, (IMPRS-PhD-Fellowship-Program and ELLIS-PhD-Fellowship-Program) (Published) BibTeX

Empirical Inference Article Real-time inference for binary neutron star mergers using machine learning Dax, M., Green, S. R., Gair, J., Gupte, N., Pürrer, M., Raymond, V., Wildberger, J., Macke, J. H., Buonanno, A., Schölkopf, B. Nature, 639(8053):49-53, March 2025 (Published) DOI URL BibTeX

Empirical Inference Article Artificial intelligence for modelling infectious disease epidemics Kraemer, M. U. G., Tsui, J. L., Chang, S. Y., Lytras, S., Khurana, M. P., Vanderslott, S., Bajaj, S., Scheidwasser, N., Curran-Sebastian, J. L., Semenova, E., Zhang, M., Unwin, H. J. T., Watson, O. J., Mills, C., Dasgupta, A., Ferretti, L., Scarpino, S. V., Koua, E., Morgan, O., Tegally, H., et al. Nature, 638(8051):623-635, February 2025 (Published) DOI URL BibTeX

Empirical Inference Ph.D. Thesis Predictions, Policies, Rewards: Models of Decision-Making from Observational Data Pace, A. ETH Zurich, Switzerland, February 2025, ETH AI Center-Fellowship-Program (Published) BibTeX

Empirical Inference Article Flow Matching for Atmospheric Retrieval of Exoplanets: Where Reliability meets Adaptive Noise Levels Gebhard, T. D., Wildberger, J., Dax, M., Kofler, A., Angerhausen, D., Quanz, S., Schölkopf, B. Astronomy \& Astrophysics, 693, January 2025 (Published) DOI URL BibTeX

Empirical Inference Ph.D. Thesis Machine Learning Meets Exoplanet Science: Methodical contributions to direct imaging and atmospheric retrieval Gebhard, T. ETH Zurich, Switzerland, January 2025 (Published) BibTeX

Empirical Inference Technical Report International AI Safety Report Bengio, Y., Mindermann, S., Privitera, D., Besiroglu, T., Bommasani, R., Casper, S., Choi, Y., Fox, P., Garfinkel, B., Goldfarb, D., Heidari, H., Ho, A., Kapoor, S., Khalatbari, L., Longpre, S., Manning, S., Mavroudis, V., Mazeika, M., Michael, J., Newman, J., et al. (DSIT 2025/001), 2025 (Published) URL BibTeX

Empirical Inference Book Chapter Natural Language Processing Jin, Z., Mihalcea, R., Schölkopf, B. In Elgar Encyclopedia of Political Communication, (Editors: Nai, A. and Grömping, M. and Wirz, D.), Edward Elgar Publishing, 2025 (Published) PDF URL BibTeX

Empirical Inference Conference Paper From Causal to Concept-Based Representation Learning Rajendran*, G., Buchholz*, S., Aragam, B., Schölkopf, B., Ravikumar, P. K. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:101250-101296, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Learning Partitions from Context Buchholz, S. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:140066-140112, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving Didolkar, A. R., Goyal, A., Ke, N. R., Guo, S., Valko, M., Lillicrap, T. P., Rezende, D. J., Bengio, Y., Mozer, M. C., Arora, S. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:19783-19812, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper A Generative Model of Symmetry Transformations Allingham, J. U., Mlodozeniec, B. K., Padhy, S., Antorán, J., Krueger, D., Turner, R. E., Nalisnick, E., Hernández-Lobato, J. M. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:91091-91130, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Article A Randomized Controlled Trial on Anonymizing Reviewers to Each Other in Peer Review Discussions Rastogi, C., Song, X., Jin, Z., Stelmakh, I., Daumé III, H., Zhang, K., Shah, N. B. PLOS ONE, 19(12), Public Library of Science, December 2024 (Published) DOI URL BibTeX

Empirical Inference Conference Paper Alien Recombination: Exploring Concept Blends Beyond Human Cognitive Availability in Visual Art Hernandez, A., Brinkmann, L., Serna, I., Rahaman, N., Alhaija, H. A., Yakura, H., Sola, M. C., Schölkopf, B., Rahwan, I. NeurIPS 2024 Workshop on Creativity and Generative AI, December 2024 (Published) arXiv BibTeX

Empirical Inference Conference Paper Causal vs. Anticausal merging of predictors Garrido Mejia, S., Blöbaum, P., Schölkopf, B., Janzing, D. Advances in Neural Information Processing Systems 37 (NeurIPS 2024) , 37:1402-1427, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Ph.D. Thesis Causality for Natural Language Processing Jin, Z. University of Tübingen, Germany, December 2024, (ELLIS PhD student program) (Published) URL BibTeX

Empirical Inference Conference Paper Causally Testing Gender Bias in LLMs: A Case Study on Occupational Bias Chen*, Y., Vethavikashini*, C. R., Mattern*, J., Mihalcea, R., Jin, Z. NeurIPS 2024 Workshop on Causality and Language Models (CaLM), December 2024, *equal contribution (Published) DOI URL BibTeX

Empirical Inference Conference Paper Cooperate or Collapse: Emergence of Sustainability in a Society of LLM Agents Piatti*, G., Jin*, Z., Kleiman-Weiner*, M., Schölkopf, B., Sachan, M., Mihalcea, R. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:111715-111759, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024, *equal contribution (Published) arXiv URL BibTeX

Empirical Inference Conference Paper Do Finetti: On Causal Effects for Exchangeable Data Guo, S., Zhang, C., Muhan, K., Huszár*, F., Schölkopf*, B. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:127317-127345, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024, *equal supervision (Published) URL BibTeX

Empirical Inference Conference Paper Improving Linear System Solvers for Hyperparameter Optimisation in Iterative Gaussian Processes Lin, J. A., Padhy, S., Mlodozeniec, B. K., Antorán, J., Hernández-Lobato, J. M. Advances in Neural Information Processing Systems 37 (NeurIPS 2024) , 37:15460-15496, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Inferring stochastic low-rank recurrent neural networks from neural data Pals, M., Sağtekin, A. E., Pei, F., Gloeckler, M., Macke, J. Advances in Neural Information Processing Systems 37 (NeurIPS 2024) , 37:18225-18264, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Latent Diffusion for Neural Spiking Data Kapoor, J., Schulz, A., Vetter, J., Pei, F., Gao, R., Macke, J. H. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:118119-118154, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Limits of Transformer Language Models on Learning to Compose Algorithms Thomm, J., Camposampiero, G., Terzic, A., Hersche, M., Schölkopf, B., Rahimi, A. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:7631-7674, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) arXiv URL BibTeX

Empirical Inference Conference Paper Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks Chen, W., Ge, H. Advances in Neural Information Processing Systems 37 (NeurIPS 2024) , 37:97562-97586, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper On Affine Homotopy between Language Encoders Chan, R., Bourmasmoud, R., Svete, A., Ren, Y., Guo, Q., Jin, Z., Ravfogel, S., Sachan, M., Schölkopf, B., El-Assady, M., Cotterell, R. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:73337-73365, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Shaving Weights with Occam’s Razor: Bayesian Sparsification for Neural Networks using the Marginal Likelihood Dhahri, R., Immer, A., Charpentier, B., Günnemann, S., Fortuin, V. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:24959-24989, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation Vetter, J., Moss, G., Schröder, C., Gao, R., Macke, J. H. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:88772-88806, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Theoretical Characterisation of the Gauss Newton Conditioning in Neural Networks Zhao*, J., Singh*, S. P., Lucchi, A. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:114965-115000, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024, *equal contribution (Published) URL BibTeX

Empirical Inference Conference Paper What Makes and Breaks Safety Fine-tuning? A Mechanistic Study Jain, S., Lubana, E. S., Oksuz, K., Joy, T., Torr, P., Sanyal, A., Dokania, P. K. Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 37:93406-93478, (Editors: A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang), Curran Associates, Inc., 38th Annual Conference on Neural Information Processing Systems, December 2024 (Published) URL BibTeX

Haptic Intelligence Autonomous Learning Empirical Inference Miscellaneous Demonstration: Minsight - A Soft Vision-Based Tactile Sensor for Robotic Fingertips Andrussow, I., Sun, H., Martius, G., Kuchenbecker, K. J. Hands-on demonstration presented at the Conference on Robot Learning (CoRL), Munich, Germany, November 2024 (Published)

Abstract ›

Beyond vision and hearing, tactile sensing enhances a robot's ability to dexterously manipulate unfamiliar objects and safely interact with humans. Giving touch sensitivity to robots requires compact, robust, affordable, and efficient hardware designs, especially for high-resolution tactile sensing. We present a soft vision-based tactile sensor engineered to meet these requirements. Comparable in size to a human fingertip, Minsight uses machine learning to output high-resolution directional contact force distributions at 60 Hz. Minsight's tactile force maps enable precise sensing of fingertip contacts, which we use in this hands-on demonstration to allow a 3-DoF robot arm to physically track contact with a user's finger. While observing the colorful image captured by Minsight's internal camera, attendees can experience how its ability to detect delicate touches in all directions facilitates real-time robot interaction.

BibTeX

Empirical Inference Conference Paper Diffusion-based learning of contact plans for agile locomotion Dh’Edin, V., Ravi, A. K. C., Jordana, A., Zhu, H., Meduri, A., Righetti, L., Schölkopf, B., Khadiv, M. IEEE-RAS 23rd International Conference on Humanoid Robots (Humanoids), 637-644, IEEE, November 2024 (Published) DOI URL BibTeX

Empirical Inference Conference Paper Do LLMs Think Fast and Slow? A Causal Study on Sentiment Analysis Lyu*, Z., Jin*, Z., Gonzalez, F., Mihalcea, R., Schölkopf, B., Sachan, M. Findings of the Association for Computational Linguistics: EMNLP, 9353-9372, (Editors: Yaser Al-Onaizan and Mohit Bansal and Yun-Nung Chen), Association for Computational Linguistics, November 2024, *equal contribution (Published) DOI URL BibTeX

Empirical Inference Conference Paper Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis Jenny*, D. F., Billeter*, Y., Schölkopf, B., Jin, Z. Proceedings of the Third Workshop on NLP for Positive Impact, 152-178, (Editors: Dementieva, Daryna and Ignat, Oana and Jin, Zhijing and Mihalcea, Rada and Piatti, Giorgio and Tetreault, Joel and Wilson, Steven and Zhao, Jieyu), Association for Computational Linguistics, November 2024, *equal contribution (Published) URL BibTeX

Empirical Inference Conference Paper Implicit Personalization in Language Models: A Systematic Study Jin, Z., Heil, N., Liu, J., Dhuliawala, S., Qi, Y., Schölkopf, B., Mihalcea, R., Sachan, M. Findings of the Association for Computational Linguistics: EMNLP, 12309-12325, (Editors: Yaser Al-Onaizan and Mohit Bansal and Yun-Nung Chen), Association for Computational Linguistics, November 2024 (Published) DOI URL BibTeX

Empirical Inference Ph.D. Thesis On Principled Modeling of Inductive Bias in Machine Learning Liu, W. University of Cambridge, UK, Cambridge, November 2024, (Cambridge-Tübingen-Fellowship-Program, ELLIS PhD student program) (Published) BibTeX

Empirical Inference Conference Paper The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoning Cui, S., Jin, Z., Schölkopf, B., Faltings, B. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 16722-16763, (Editors: Al-Onaizan, Yaser and Bansal, Mohit and Chen, Yun-Nung), Association for Computational Linguistics, November 2024 (Published) URL BibTeX

Empirical Inference Conference Paper RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands Zhao*, Y., Chen*, L., Schneider, J., Gao, Q., Kannala, J., Schölkopf, B., Pajarinen, J., Büchler, D. Proceedings of the 8th Annual Conference on Robot Learning (CoRL), 270:5184-5203, Proceedings of Machine Learning Research, (Editors: Agrawal, Pulkit and Kroemer, Oliver and Burgard, Wolfram), PMLR, Conference on Robot Learning, November 2024, *equal contribution (Published) URL BibTeX

Empirical Inference Article A Probabilistic Model behind Self-Supervised Learning Bizeul, A., Schölkopf, B., Allen, C. Transactions on Machine Learning Research, October 2024 (Published) PDF URL BibTeX

Empirical Inference Article How developments in natural language processing help us in understanding human behaviour Mihalcea, R., Biester, L., Boyd, R. L., Jin, Z., Perez-Rosas, V., Wilson, S., Pennebaker, J. W. Nature Human Behaviour, 8(10):1877-1889, Nature Publishing Group UK London, October 2024 (Published) DOI URL BibTeX

Empirical Inference Conference Paper Redesigning Information Markets in the Era of Language Models Weiss, M., Rahaman, N., Wüthrich, M., Bengio, Y., Li, L. E., Schölkopf, B., Pal, C. First Conference on Language Modeling (COLM), arXiv:2403.14443, October 2024 (Published)

Abstract ›

This work addresses the buyer's inspection paradox for information markets. The paradox is that buyers need to access information to determine its value, while sellers need to limit access to prevent theft. To study this, we introduce an open-source simulated digital marketplace where intelligent agents, powered by language models, buy and sell information on behalf of external participants. The central mechanism enabling this marketplace is the agents' dual capabilities: they not only have the capacity to assess the quality of privileged information but also come equipped with the ability to forget. This ability to induce amnesia allows vendors to grant temporary access to proprietary information, significantly reducing the risk of unauthorized retention while enabling agents to accurately gauge the information's relevance to specific queries or tasks. To perform well, agents must make rational decisions, strategically explore the marketplace through generated sub-queries, and synthesize answers from purchased information. Concretely, our experiments (a) uncover biases in language models leading to irrational behavior and evaluate techniques to mitigate these biases, (b) investigate how price affects demand in the context of informational goods, and (c) show that inspection and higher budgets both lead to higher quality outcomes.

arXiv URL BibTeX

Empirical Inference Conference Paper GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs Gao, G., Liu, W., Chen, A., Geiger, A., Schölkopf, B. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 21295-21304, IEEE, Piscataway, NJ, CVPR, September 2024 (Published) DOI URL BibTeX

Publications

Filter by