Publications

Empirical Inference Conference Paper Certified private data release for sparse Lipschitz functions Donhauser, K., Lokna, J., Sanyal, A., Boedihardjo, M., Hönig, R., Yang, F. Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), 238:1396-1404, Proceedings of Machine Learning Research, (Editors: Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen), PMLR, May 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding Pace, A., Yèche, H., Schölkopf, B., Rätsch, G., Tennenholtz, G. The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published) arXiv BibTeX

Perceiving Systems Empirical Inference Conference Paper Ghost on the Shell: An Expressive Representation of General 3D Shapes Liu, Z., Feng, Y., Xiu, Y., Liu, W., Paull, L., Black, M. J., Schölkopf, B. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published)

Abstract ›

The creation of photorealistic virtual worlds requires the accurate modeling of 3D surface geometry for a wide range of objects. For this, meshes are appealing since they 1) enable fast physics-based rendering with realistic material and lighting, 2) support physical simulation, and 3) are memory-efficient for modern graphics pipelines. Recent work on reconstructing and statistically modeling 3D shape, however, has critiqued meshes as being topologically inflexible. To capture a wide range of object shapes, any 3D representation must be able to model solid, watertight, shapes as well as thin, open, surfaces. Recent work has focused on the former, and methods for reconstructing open surfaces do not support fast reconstruction with material and lighting or unconditional generative modelling. Inspired by the observation that open surfaces can be seen as islands floating on watertight surfaces, we parameterize open surfaces by defining a manifold signed distance field on watertight templates. With this parameterization, we further develop a grid-based and differentiable representation that parameterizes both watertight and non-watertight meshes of arbitrary topology. Our new representation, called Ghost-on-the-Shell (G-Shell), enables two important applications: differentiable rasterization-based reconstruction from multiview images and generative modelling of non-watertight meshes. We empirically demonstrate that G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks, while also performing effectively for watertight meshes.

Home Code Video Project BibTeX

Empirical Inference Article Grundfragen der künstlichen Intelligenz Schölkopf, B. astronomie - Das Magazin, 42, May 2024 (Published) URL BibTeX

Empirical Inference Conference Paper Identifying Policy Gradient Subspaces Schneider, J., Schumacher, P., Guist, S., Chen, L., Häufle, D., Schölkopf, B., Büchler, D. The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published) arXiv BibTeX

Empirical Inference Autonomous Learning Conference Paper Multi-View Causal Representation Learning with Partial Observability Yao, D., Xu, D., Lachapelle, S., Magliacane, S., Taslakian, P., Martius, G., von Kügelgen, J., Locatello, F. The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published) arXiv BibTeX

Empirical Inference Conference Paper Open X-Embodiment: Robotic Learning Datasets and RT-X Models Open X-Embodiment Collaboration ( incl. Guist, S., Schneider, J., Schölkopf, B., Büchler, D. ). IEEE International Conference on Robotics and Automation (ICRA), 6892-6903, May 2024 (Published) arXiv DOI URL BibTeX

Empirical Inference Conference Paper Out-of-Variable Generalization for Discriminative Models Guo, S., Wildberger, J., Schölkopf, B. The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published) arXiv BibTeX

Empirical Inference Perceiving Systems Conference Paper Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Liu, W., Qiu, Z., Feng, Y., Xiu, Y., Xue, Y., Yu, L., Feng, H., Liu, Z., Heo, J., Peng, S., Wen, Y., Black, M. J., Weller, A., Schölkopf, B. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), The Twelfth International Conference on Learning Representations, May 2024 (Published)

Abstract ›

Large foundation models are becoming ubiquitous, but training them from scratch is prohibitively expensive. Thus, efficiently adapting these powerful models to downstream tasks is increasingly important. In this paper, we study a principled finetuning paradigm -- Orthogonal Finetuning (OFT) -- for downstream task adaptation. Despite demonstrating good generalizability, OFT still uses a fairly large number of trainable parameters due to the high dimensionality of orthogonal matrices. To address this, we start by examining OFT from an information transmission perspective, and then identify a few key desiderata that enable better parameter-efficiency. Inspired by how the Cooley-Tukey fast Fourier transform algorithm enables efficient information transmission, we propose an efficient orthogonal parameterization using butterfly structures. We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT). By subsuming OFT as a special case, BOFT introduces a generalized orthogonal finetuning framework. Finally, we conduct an extensive empirical study of adapting large vision transformers, large language models, and text-to-image diffusion models to various downstream tasks in vision and language.

Home Code HuggingFace project URL BibTeX

Empirical Inference Conference Paper Skill or Luck? Return Decomposition via Advantage Functions Pan, H., Schölkopf, B. The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published) arXiv BibTeX

Empirical Inference Conference Paper Some Intriguing Aspects about Lipschitz Continuity of Neural Networks Khromov*, G., Singh*, S. P. The Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (Published) arXiv BibTeX

Empirical Inference Conference Paper Stochastic Gradient Descent for Gaussian Processes Done Right Lin*, J. A., Padhy*, S., Antorán*, J., Tripp, A., Terenin, A., Szepesvari, C., Hernández-Lobato, J. M., Janz, D. The Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (Published) arXiv BibTeX

Empirical Inference Conference Paper Targeted Reduction of Causal Models Kekić, A., Schölkopf, B., Besserve, M. ICLR 2024 Workshop on AI4DifferentialEquations In Science, May 2024 (Published) URL BibTeX

Empirical Inference Autonomous Learning Conference Paper The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks Spieler, A., Rahaman, N., Martius, G., Schölkopf, B., Levina, A. In The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published) arXiv BibTeX

Empirical Inference Conference Paper Towards Meta-Pruning via Optimal Transport Theus, A., Geimer, O., Wicke, F., Hofmann, T., Anagnostidis, S., Singh, S. P. The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published) arXiv BibTeX

Empirical Inference Conference Paper Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion Meterez*, A., Joudaki*, A., Orabona, F., Immer, A., Rätsch, G., Daneshmand, H. The Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (Published) arXiv BibTeX

Empirical Inference Conference Paper Transformer Fusion with Optimal Transport Imfeld*, M., Graldi*, J., Giordano*, M., Hofmann, T., Anagnostidis, S., Singh, S. P. The Twelfth International Conference on Learning Representations (ICLR), May 2024, *equal contribution (Published) arXiv BibTeX

Empirical Inference Master Thesis Algorithmic Compositional Learning of Language Models Thomm, J. ETH Zurich, Switzerland, April 2024 (Published) BibTeX

Empirical Inference Miscellaneous Evidence for eccentricity in the population of binary black holes observed by LIGO-Virgo-KAGRA Gupte, N., Ramos-Buades, A., Buonanno, A., Gair, J., Miller, M. C., Dax, M., Green, S. R., Pürrer, M., Wildberger, J., Macke, J. H., Romero-Shaw, I. M., Schölkopf, B. April 2024 (Published) URL BibTeX

Empirical Inference Conference Paper PILLAR: How to make semi-private learning more effective Pinto, F., Hu, Y., Yang, F., Sanyal, A. 2nd IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 110-139, April 2024 (Published) DOI BibTeX

Empirical Inference Article SimReadUntil for benchmarking selective sequencing algorithms on ONT devices Mordig, M., Ratsch, G., Kahles, A. Bioinformatics, 40(5):btae199, April 2024 (Published) DOI URL BibTeX

Empirical Inference Article VIPurPCA: Visualizing and Propagating Uncertainty in Principal Component Analysis Zabel, S., Hennig, P., Nieselt, K. IEEE Transactions on Visualization and Computer Graphics, 30(4):2011-2022, April 2024 (Published) DOI BibTeX

Empirical Inference Poster Koopman Spectral Analysis Uncovers the Temporal Structure of Spontaneous Neural Events Shao, K., Xu, Y., Logothetis, N., Shen, Z., Besserve, M. Computational and Systems Neuroscience Meeting (COSYNE), March 2024 (Published) URL BibTeX

Empirical Inference Master Thesis Interpreting How Large Language Models Handle Facts and Counterfactuals through Mechanistic Interpretability Ortu, F. University of Trieste, Italy, March 2024 (Published) BibTeX

Empirical Inference Article Learning Graph Embeddings for Open World Compositional Zero-Shot Learning Mancini, M., Naeem, M. F., Xian, Y., Akata, Z. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(3):1545-1560, IEEE, New York, NY, March 2024 (Published) DOI BibTeX

Empirical Inference Poster LFP transient events in macaque subcortical areas reveal network coordination across scales and structures: a simultaneous fMRI-electrophysiology study Besserve, M., Safavi, S., Schölkopf, B., Logothetis, N. Computational and Systems Neuroscience Meeting (COSYNE), March 2024 (Published) URL BibTeX

Empirical Inference Ph.D. Thesis Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment von Kügelgen, J. University of Cambridge, UK, Cambridge, February 2024, (Cambridge-Tübingen-Fellowship) (Published) URL BibTeX

Empirical Inference Conference Paper Inferring Atmospheric Properties of Exoplanets with Flow Matching and Neural Importance Sampling Gebhard, T. D., Wildberger, J., Dax, M., Angerhausen, D., Quanz, S. P., Schölkopf, B. 3rd Annual AAAI Workshop on AI to Accelerate Science and Engineering (AI2ASE), February 2024 (Published) arXiv URL BibTeX

Empirical Inference Ph.D. Thesis Methods for Generative Modeling and Interpretable Classification with Strong Differential Privacy Harder, F. University of Tübingen, Germany, February 2024, IMPRS-PhD-Program (Published) URL BibTeX

Empirical Inference Article Trained recurrent neural networks develop phase-locked limit cycles in a working memory task Pals, M., Macke, J. H., Barak, O. PLOS Computational Biology, 20(2), February 2024 (Published) DOI BibTeX

Empirical Inference Conference Paper Multi-channel free space optical convolutions with incoherent light Song, A., Kottapalli, S. N. M., Schölkopf, B., Fischer, P. AI and Optical Data Sciences V, PC12903:PC129030I, (Editors: Ken-ichi Kitayama and Volker J. Sorger), SPIE, January 2024 (Published) DOI BibTeX

Empirical Inference Article Network propagation for GWAS analysis: a practical guide to leveraging molecular networks for disease gene discovery Visonà, G., Bouzigon, E., Demenais, F., Schweikert, G. Briefings in Bioinformatics, 25(2), January 2024 (Published) DOI BibTeX

Empirical Inference Optics and Sensing Laboratory Conference Paper Polarization-based non-linear deep diffractive neural networks Kottapalli, S. N. M., Schlieder, L., Song, A., Volchkov, V., Schölkopf, B., Fischer, P. AI and Optical Data Sciences V, PC12903:PC129030B, (Editors: Ken-ichi Kitayama and Volker J. Sorger), SPIE, January 2024 (Published) DOI BibTeX

Empirical Inference Article Pre-treatment 18F-FDG-PET/CT parameters as biomarkers for progression free survival, best overall response and overall survival in metastatic melanoma patients undergoing first-line immunotherapy Peisen, F., Gerken, A., Dahm, I., Nikolaou, K., Eigentler, T., Amaral, T., Moltz, J. H., Othman, A. E., Gatidis, S. PLOS ONE, 19(1), January 2024 (Published) DOI BibTeX

Empirical Inference Article Towards fully covariant machine learning Villar, S., Hogg, D. W., Yao, W., Kevrekidis, G. A., Schölkopf, B. Transactions on Machine Learning Research, January 2024 (Published) URL BibTeX

Empirical Inference Miscellaneous Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models Rajendran, G., Buchholz, S., Aragam, B., Schölkopf, B., Ravikumar, P. 2024 (Published) arXiv BibTeX

Empirical Inference Miscellaneous Multi-Armed Bandits and Quantum Channel Oracles Buchholz, S., Kübler, J. M., Schölkopf, B. 2024 (Published) arXiv BibTeX

Empirical Inference Article Optimal Decision Making Under Strategic Behavior Tsirtsis, S., Tabibian, B., Khajehnejad, M., Singla, A., Schölkopf, B., Gomez-Rodriguez, M. Management Science, 2024, Published Online (In press) DOI BibTeX

Empirical Inference Article Parameterizing pressure-temperature profiles of exoplanet atmospheres with neural networks Gebhard, T. D., Angerhausen, D., Konrad, B. S., Alei, E., Quanz, S. P., Schölkopf, B. Astronomy & Astrophysics, 681, 2024 (Published) DOI BibTeX

Empirical Inference Article Use the 4S (Signal-Safe Speckle Subtraction): Explainable Machine Learning reveals the Giant Exoplanet AF Lep b in High-Contrast Imaging Data from 2011 Bonse, M. J., Gebhard, T. D., Dannert, F. A., Absil, O., Cantalloube, F., Christiaens, V., Cugno, G., Garvin, E. O., Hayoz, J., Kasper, M., Matthews, E., Schölkopf, B., Quanz, S. P. The Astronomical Journal, 2024 (Accepted) arXiv BibTeX

Empirical Inference Article A phenomenological account for causality in terms of elementary actions Janzing, D., Garrido Mejia, S. Journal of Causal Inference, 12(1), 2024 (Published) DOI URL BibTeX

Empirical Inference Article A temperate super-Jupiter imaged with JWST in the mid-infrared Matthews, E. C., Carter, A. L., Pathak, P., Morley, C. V., Phillips, M. W., S. Krishanth, P. M., Feng, F., Bonse, M. J., Boogaard, L. A., Burt, J. A., Crossfield, I. J. M., Douglas, E. S., Henning, T., Hom, J., Ko, C. -., Kasper, M., Lagrange, A., Petit Dit de la Roche, D., Philipot, F. Nature, 633:789–792, 2024 (Published)

Abstract ›

Of the approximately 25 directly imaged planets to date, all are younger than 500 Myr, and all but six are younger than 100 Myr (ref. 1). Eps Ind A (HD209100, HIP108870) is a K5V star of roughly solar age (recently derived as 3.7–5.7 Gyr (ref. 2) and Gyr (ref. 3)). A long-term radial-velocity trend4,5 and an astrometric acceleration6,7 led to claims of a giant planet2,8,9 orbiting the nearby star (3.6384 ± 0.0013 pc; ref. 10). Here we report JWST coronagraphic images which reveal a giant exoplanet that is consistent with these radial and astrometric measurements but inconsistent with the previously claimed planet properties. The new planet has a temperature of approximately 275 K and is remarkably bright at 10.65 and 15.50 µm. Non-detections between 3.5 and 5.0 µm indicate an unknown opacity source in the atmosphere, possibly suggesting a high-metallicity, high carbon-to-oxygen ratio planet. The best-fitting temperature of the planet is consistent with theoretical thermal evolution models, which were previously untested at this temperature range. The data indicate that this is probably the only giant planet in the system, and therefore we refer to it as b, despite it having significantly different orbital properties than the previously claimed planet b.

DOI URL BibTeX

Empirical Inference Miscellaneous Analyzing Human Questioning Behavior and Causal Curiosity through Natural Queries Ceraolo, R., Kharlapenko, D., Khan, A., Reymond, A., Mihalcea, R., Sachan, M., Schölkopf, B., Jin, Z. 2024 (Published) URL BibTeX

Empirical Inference Article Connectome-constrained networks predict neural activity across the fly visual system Lappalainen, J. K., Tschopp, F. D., Prakhya, S., McGill, M., Nern, A., Shinomiya, K., Takemura, S., Gruntman, E., Macke, J. H., Turaga, S. C. Nature, 634:1132–1140, 2024 (Published)

Abstract ›

We can now measure the connectivity of every neuron in a neural circuit, but we cannot measure other biological details, including the dynamical characteristics of each neuron. The degree to which measurements of connectivity alone can inform the understanding of neural computation is an open question10. Here we show that with experimental measurements of only the connectivity of a biological neural network, we can predict the neural activity underlying a specified neural computation. We constructed a model neural network with the experimentally determined connectivity for 64 cell types in the motion pathways of the fruit fly optic lobe but with unknown parameters for the single-neuron and single-synapse properties. We then optimized the values of these unknown parameters using techniques from deep learning, to allow the model network to detect visual motion. Our mechanistic model makes detailed, experimentally testable predictions for each neuron in the connectome. We found that model predictions agreed with experimental measurements of neural activity across 26 studies. Our work demonstrates a strategy for generating detailed hypotheses about the mechanisms of neural circuit function from connectivity measurements. We show that this strategy is more likely to be successful when neurons are sparsely connected—a universally observed feature of biological neural networks across species and brain regions.

DOI URL BibTeX

Empirical Inference Conference Paper DeViL: Decoding Vision features into Language Dani, M., Rio-Torto, I., Alaniz, S., Akata, Z. In Lecture Notes in Computer Science, vol 14264, 363–377, 45th Annual Conference of the German-Association-for-Pattern-Recognition (DAGM GCPR), 2024 (Published) DOI URL BibTeX

Empirical Inference Miscellaneous Language Model Alignment in Multilingual Trolley Problems Jin, Z., Levine, S., Kleiman-Weiner, M., Piatti, G., Liu, J., Gonzalez, F., Ortu, F., Strausz, A., Sachan, M., Mihalcea, R., Choi, Y., Schölkopf, B. 2024 (Published) URL BibTeX

Empirical Inference Article Low-power scalable multilayer optoelectronic neural networks enabled with incoherent light Song, A., Kottapalli, S. N. M., Goyal, R., Schoelkopf, B., Fischer, P. Nature Communications, 15:10692, 2024 (Published)

Abstract ›

Optical approaches have made great strides towards the goal of high-speed, energy-efficient computing necessary for modern deep learning and AI applications. Read-in and read-out of data, however, limit the overall performance of existing approaches. This study introduces a multilayer optoelectronic computing framework that alternates between optical and optoelectronic layers to implement matrix-vector multiplications and rectified linear functions, respectively. Our framework is designed for real-time, parallelized operations, leveraging 2D arrays of LEDs and photodetectors connected via independent analog electronics. We experimentally demonstrate this approach using a system with a three-layer network with two hidden layers and operate it to recognize images from the MNIST database with a recognition accuracy of 92% and classify classes from a nonlinear spiral data with 86% accuracy. By implementing multiple layers of a deep neural network simultaneously, our approach significantly reduces the number of read-ins and read-outs required and paves the way for scalable optical accelerators requiring ultra low energy.

DOI URL BibTeX

Empirical Inference Article Neonatal apnea and hypopnea prediction in infants with Robin sequence with neural additive models for time series Vetter, J., Lim, K., Dijkstra, T. M. H., Dargaville, P. A., Kohlbacher, O., Macke, J. H., Poets, C. F. PLOS Health Digital, 3(12):e0000678, 2024 (Accepted) DOI URL BibTeX

Empirical Inference Conference Paper Waffling around for Performance: Visual Classification with Random Words and Broad Concepts Roth, K., Kim, J. M., Koepke, A. S., Vinyals, O., Schmid, C., Akata, Z. In Proceedings 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 15700-15711, 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2024 (Published) DOI URL BibTeX

Empirical Inference Conference Paper Optimistic Active Exploration of Dynamical Systems Sukhija, B., Treven, L., Sancaktar, C., Blaes, S., Coros, S., Krause, A. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 1-32, Curran Associates Inc. , NeurIPS, December 2023 (Published) DOI URL BibTeX

Publications

Filter by