Publications

Empirical Inference Conference Paper Forecasting in Offline Reinforcement Learning for Non-stationary Environments Ada, S. E., Martius, G., Ugur, E., Oztop, E. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025), 39th Annual Conference on Neural Information Processing Systems, December 2025 (Accepted) arXiv BibTeX

Haptic Intelligence Autonomous Learning Empirical Inference Conference Paper Adding Internal Audio Sensing to Internal Vision Enables Human-Like In-Hand Fabric Recognition with Soft Robotic Fingertips Andrussow, I., Solano, J., Richardson, B. A., Martius, G., Kuchenbecker, K. J. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots (Humanoids), 373-380, Seoul, South Korea, September 2025 (Published)

Abstract ›

Distinguishing the feel of smooth silk from coarse cotton is a trivial everyday task for humans. When exploring such fabrics, fingertip skin senses both spatio-temporal force patterns and texture-induced vibrations that are integrated to form a haptic representation of the explored material. It is challenging to reproduce this rich, dynamic perceptual capability in robots because tactile sensors typically cannot achieve both high spatial resolution and high temporal sampling rate. In this work, we present a system that can sense both types of haptic information, and we investigate how each type influences robotic tactile perception of fabrics. Our robotic hand's middle finger and thumb each feature a soft tactile sensor: one is the open- source Minsight sensor that uses an internal camera to measure fingertip deformation and force at 50 Hz, and the other is our new sensor Minsound that captures vibrations through an internal MEMS microphone with a bandwidth from 50 Hz to 15 kHz. Inspired by the movements humans make to evaluate fabrics, our robot actively encloses and rubs folded fabric samples between its two sensitive fingers. Our results test the influence of each sensing modality on overall classification performance, showing high utility for the audio-based sensor. Our transformer-based method achieves a maximum fabric classification accuracy of 97% on a dataset of 20 common fabrics. Incorporating an external microphone away from Minsound increases our method's robustness in loud ambient noise conditions. To show that this audio-visual tactile sensing approach generalizes beyond the training data, we learn general representations of fabric stretchiness, thickness, and roughness.

DOI BibTeX

Empirical Inference Conference Paper Active Fine-Tuning of Multi-Task Policies Bagatella, M., Hübotter, J., Martius, G., Krause, A. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 267:2409-2441, Proceedings of Machine Learning Research, (Editors: Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry), PMLR, International Conference on Machine Learning, July 2025 (Published) arXiv URL BibTeX

Empirical Inference Autonomous Learning Conference Paper SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Models Sancaktar, C., Gumbsch, C., Zadaianchuk, A., Kolev, P., Martius, G. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 267:52745-52777, Proceedings of Machine Learning Research, (Editors: Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry), International Conference on Machine Learning , July 2025 (Published) arXiv Project website URL BibTeX

Autonomous Learning Empirical Inference Conference Paper Zero-Shot Offline Imitation Learning via Optimal Transport Rupf, T., Bagatella, M., Gürtler, N., Frey, J., Martius, G. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 267:52345-52381, Proceedings of Machine Learning Research, (Editors: Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry), PMLR, International Conference on Machine Learning, July 2025 (Published)

Abstract ›

Zero-shot imitation learning algorithms hold the promise of reproducing unseen behavior from as little as a single demonstration at test time. Existing practical approaches view the expert demonstration as a sequence of goals, enabling imitation with a high-level goal selector, and a low-level goal-conditioned policy. However, this framework can suffer from myopic behavior: the agent's immediate actions towards achieving individual goals may undermine long-term objectives. We introduce a novel method that mitigates this issue by directly optimizing the occupancy matching objective that is intrinsic to imitation learning. We propose to lift a goal-conditioned value function to a distance between occupancies, which are in turn approximated via a learned world model. The resulting method can learn from offline, suboptimal data, and is capable of non-myopic, zero-shot imitation, as we demonstrate in complex, continuous benchmarks.

arXiv URL BibTeX

Empirical Inference Conference Paper Temporally Consistent Object-Centric Learning by Contrasting Slots Manasyan, A., Seitzer, M., Radovic, F., Martius, G., Zadaianchuk, A. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 5401-5411, June 2025 (Published) DOI BibTeX

Empirical Inference Conference Paper VERA: Explainable Video Anomaly Detection via Verbalized Learning of Vision-Language Models Ye, M., Liu, W., He, P. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8679-8688, June 2025 (Published) DOI BibTeX

Empirical Inference Perceiving Systems Conference Paper ChatHuman: Chatting about 3D Humans with Tools Lin, J., Feng, Y., Liu, W., Black, M. J. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8150-8161, June 2025 (Published)

Abstract ›

Numerous methods have been proposed to detect, estimate, and analyze properties of people in images, including 3D pose, shape, contact, human-object interaction, and emotion. While widely applicable in vision and other areas, such methods require expert knowledge to select, use, and interpret the results. To address this, we introduce ChatHuman, a language-driven system that integrates the capabilities of specialized methods into a unified framework. ChatHuman functions as an assistant proficient in utilizing, analyzing, and interacting with tools specific to 3D human tasks, adeptly discussing and resolving related challenges. Built on a Large Language Model (LLM) framework, ChatHuman is trained to autonomously select, apply, and interpret a diverse set of tools in response to user inputs. Our approach overcomes significant hurdles in adapting LLMs to 3D human tasks, including the need for domain-specific knowledge and the ability to interpret complex 3D outputs. The innovations of ChatHuman include leveraging academic publications to instruct the LLM on tool usage, employing a retrieval-augmented generation model to create in-context learning examples for managing new tools, and effectively discriminating between and integrating tool results by transforming specialized 3D outputs into comprehensible formats. Experiments demonstrate that ChatHuman surpasses existing models in both tool selection accuracy and overall performance across various 3D human tasks, and it supports interactive chatting with users. ChatHuman represents a significant step toward consolidating diverse analytical methods into a unified, robust system for 3D human tasks.

project pdf Paper DOI BibTeX

Empirical Inference Autonomous Learning Conference Paper Learning to Control Emulated Muscles in Real Robots: A Software Test Bed for Bio-Inspired Actuators in Hardware Schumacher, P., Krause, L., Schneider, J., Büchler, D., Martius, G., Haeufle, D. In Proceedings 10th International Conference on Biomedical Robotics and Biomechatronics (BioRob), 806-813, IEEE, 10th International Conference on Biomedical Robotics and Biomechatronics (BioRob), September 2024 (Published) arXiv DOI URL BibTeX

Perceiving Systems Empirical Inference Conference Paper Ghost on the Shell: An Expressive Representation of General 3D Shapes Liu, Z., Feng, Y., Xiu, Y., Liu, W., Paull, L., Black, M. J., Schölkopf, B. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published)

Abstract ›

The creation of photorealistic virtual worlds requires the accurate modeling of 3D surface geometry for a wide range of objects. For this, meshes are appealing since they 1) enable fast physics-based rendering with realistic material and lighting, 2) support physical simulation, and 3) are memory-efficient for modern graphics pipelines. Recent work on reconstructing and statistically modeling 3D shape, however, has critiqued meshes as being topologically inflexible. To capture a wide range of object shapes, any 3D representation must be able to model solid, watertight, shapes as well as thin, open, surfaces. Recent work has focused on the former, and methods for reconstructing open surfaces do not support fast reconstruction with material and lighting or unconditional generative modelling. Inspired by the observation that open surfaces can be seen as islands floating on watertight surfaces, we parameterize open surfaces by defining a manifold signed distance field on watertight templates. With this parameterization, we further develop a grid-based and differentiable representation that parameterizes both watertight and non-watertight meshes of arbitrary topology. Our new representation, called Ghost-on-the-Shell (G-Shell), enables two important applications: differentiable rasterization-based reconstruction from multiview images and generative modelling of non-watertight meshes. We empirically demonstrate that G-Shell achieves state-of-the-art performance on non-watertight mesh reconstruction and generation tasks, while also performing effectively for watertight meshes.

Home Code Video Project BibTeX

Empirical Inference Perceiving Systems Conference Paper Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization Liu, W., Qiu, Z., Feng, Y., Xiu, Y., Xue, Y., Yu, L., Feng, H., Liu, Z., Heo, J., Peng, S., Wen, Y., Black, M. J., Weller, A., Schölkopf, B. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR), The Twelfth International Conference on Learning Representations, May 2024 (Published)

Abstract ›

Large foundation models are becoming ubiquitous, but training them from scratch is prohibitively expensive. Thus, efficiently adapting these powerful models to downstream tasks is increasingly important. In this paper, we study a principled finetuning paradigm -- Orthogonal Finetuning (OFT) -- for downstream task adaptation. Despite demonstrating good generalizability, OFT still uses a fairly large number of trainable parameters due to the high dimensionality of orthogonal matrices. To address this, we start by examining OFT from an information transmission perspective, and then identify a few key desiderata that enable better parameter-efficiency. Inspired by how the Cooley-Tukey fast Fourier transform algorithm enables efficient information transmission, we propose an efficient orthogonal parameterization using butterfly structures. We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT). By subsuming OFT as a special case, BOFT introduces a generalized orthogonal finetuning framework. Finally, we conduct an extensive empirical study of adapting large vision transformers, large language models, and text-to-image diffusion models to various downstream tasks in vision and language.

Home Code HuggingFace project URL BibTeX

Empirical Inference Autonomous Learning Conference Paper The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks Spieler, A., Rahaman, N., Martius, G., Schölkopf, B., Levina, A. In The Twelfth International Conference on Learning Representations (ICLR), May 2024 (Published) arXiv BibTeX

Empirical Inference Conference Paper DeViL: Decoding Vision features into Language Dani, M., Rio-Torto, I., Alaniz, S., Akata, Z. In Lecture Notes in Computer Science, vol 14264, 363–377, 45th Annual Conference of the German-Association-for-Pattern-Recognition (DAGM GCPR), 2024 (Published) DOI URL BibTeX

Empirical Inference Conference Paper Waffling around for Performance: Visual Classification with Random Words and Broad Concepts Roth, K., Kim, J. M., Koepke, A. S., Vinyals, O., Schmid, C., Akata, Z. In Proceedings 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 15700-15711, 2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2023), 2024 (Published) DOI URL BibTeX

Empirical Inference Conference Paper Optimistic Active Exploration of Dynamical Systems Sukhija, B., Treven, L., Sancaktar, C., Blaes, S., Coros, S., Krause, A. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 1-32, Curran Associates Inc. , NeurIPS, December 2023 (Published) DOI URL BibTeX

Empirical Inference Conference Paper Causal normalizing flows: from theory to practice Javaloy, A., Sanchez-Martin, P., Valera, I. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 36:58833-58864, (Editors: A. Oh and T. Neumann and A. Globerson and K. Saenko and M. Hardt and S. Levine), Curran Associates, Inc., 37th Annual Conference on Neural Information Processing Systems, December 2023 (Published) URL BibTeX

Empirical Inference Perceiving Systems Conference Paper Controlling Text-to-Image Diffusion by Orthogonal Finetuning Qiu*, Z., Liu*, W., Feng, H., Xue, Y., Feng, Y., Liu, Z., Zhang, D., Weller, A., Schölkopf, B. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 36:79320-79362, (Editors: A. Oh and T. Neumann and A. Globerson and K. Saenko and M. Hardt and S. Levine), Curran Associates, Inc., 37th Annual Conference on Neural Information Processing Systems , December 2023, *equal contribution (Published)

Abstract ›

Large text-to-image diffusion models have impressive capabilities in generating photorealistic images from text prompts. How to effectively guide or control these powerful models to perform different downstream tasks becomes an important open problem. To tackle this challenge, we introduce a principled finetuning method -- Orthogonal Finetuning (OFT), for adapting text-to-image diffusion models to downstream tasks. Unlike existing methods, OFT can provably preserve hyperspherical energy which characterizes the pairwise neuron relationship on the unit hypersphere. We find that this property is crucial for preserving the semantic generation ability of text-to-image diffusion models. To improve finetuning stability, we further propose Constrained Orthogonal Finetuning (COFT) which imposes an additional radius constraint to the hypersphere. Specifically, we consider two important finetuning text-to-image tasks: subject-driven generation where the goal is to generate subject-specific images given a few images of a subject and a text prompt, and controllable generation where the goal is to enable the model to take in additional control signals. We empirically show that our OFT framework outperforms existing methods in generation quality and convergence speed.

Home Code URL BibTeX

Empirical Inference Conference Paper Nonparametric Teaching for Multiple Learners Zhang, C., Cao, X., Liu, W., Tsang, I. W., Kwok, J. T. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 36:7756-7786, (Editors: A. Oh and T. Naumann and A. Globerson and K. Saenko and M. Hardt and S. Levine), Curran Associates, Inc., 37th Annual Conference on Neural Information Processing Systems, December 2023 (Published) URL BibTeX

Empirical Inference Conference Paper Spuriosity Didn’t Kill the Classifier: Using Invariant Predictions to Harness Spurious Features Eastwood*, C., Singh*, S., Nicolicioiu, A. L., Vlastelica, M., von Kügelgen, J., Schölkopf, B. In Advances in Neural Information Processing Systems 36, 36:18291-18324, (Editors: A. Oh and T. Neumann and A. Globerson and K. Saenko and M. Hardt and S. Levine), Curran Associates Inc., 37th Annual Conference on Neural Information Processing Systems, December 2023, *equal contribution (Published) URL BibTeX

Empirical Inference Perceiving Systems Conference Paper One-shot Implicit Animatable Avatars with Model-based Priors Huang, Y., Yi, H., Liu, W., Wang, H., Wu, B., Wang, W., Lin, B., Zhang, D., Cai, D. In Proc. International Conference on Computer Vision (ICCV), 8940-8951, International Conference on Computer Vision, October 2023, *equal contribution (Published)

Abstract ›

Existing neural rendering methods for creating human avatars typically either require dense input signals such as video or multi-view images, or leverage a learned prior from large-scale specific 3D human datasets such that reconstruction can be performed with sparse-view inputs. Most of these methods fail to achieve realistic reconstruction when only a single image is available. To enable the data-efficient creation of realistic animatable 3D humans, we propose ELICIT, a novel method for learning human-specific neural radiance fields from a single image. Inspired by the fact that humans can easily reconstruct the body geometry and infer the full-body clothing from a single image, we leverage two priors in ELICIT: 3D geometry prior and visual semantic prior. Specifically, ELICIT introduces the 3D body shape geometry prior from a skinned vertex-based template model (i.e., SMPL) and implements the visual clothing semantic prior with the CLIP-based pre-trained models. Both priors are used to jointly guide the optimization for creating plausible content in the invisible areas. In order to further improve visual details, we propose a segmentation-based sampling strategy that locally refines different parts of the avatar.Comprehensive evaluations on multiple popular benchmarks, including ZJU-MoCAP, Human3.6M, and DeepFashion, show that ELICIT has outperformed current state-of-the-art avatar creation methods when only a single image is available. Code will be public for reseach purpose at https://github.com/huangyangyi/ELICIT

arXiv code project DOI BibTeX

Perceiving Systems Empirical Inference Conference Paper Pairwise Similarity Learning is SimPLE Wen, Y., Liu, W., Feng, Y., Raj, B., Singh, R., Weller, A., Black, M. J., Schölkopf, B. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), International Conference on Computer Vision, October 2023 (Published)

Abstract ›

In this paper, we focus on a general yet important learning problem, pairwise similarity learning (PSL). PSL subsumes a wide range of important applications, such as open-set face recognition, speaker verification, image retrieval and person re-identification. The goal of PSL is to learn a pairwise similarity function assigning a higher similarity score to positive pairs (i.e., a pair of samples with the same label) than to negative pairs (i.e., a pair of samples with different label). We start by identifying a key desideratum for PSL, and then discuss how existing methods can achieve this desideratum. We then propose a surprisingly simple proxy-free method, called SimPLE, which requires neither feature/proxy normalization nor angular margin and yet is able to generalize well in open-set recognition. We apply the proposed method to three challenging PSL tasks: open-set face recognition, image retrieval and speaker verification. Comprehensive experimental results on large-scale benchmarks show that our method performs significantly better than current state-of-the-art methods.

URL BibTeX

Embodied Vision Learning and Dynamical Systems Empirical Inference Conference Paper Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts Achterhold, J., Tobuschat, P., Ma, H., Büchler, D., Muehlebach, M., Stueckler, J. In Conference on Learning for Dynamics and Control, 211:878-890, Proceedings of Machine Learning Research, (Editors: Nikolai Matni, Manfred Morari and George J. Pappa), PMLR, June 2023 (Published) preprint code URL BibTeX

Autonomous Learning Empirical Inference Conference Paper Benchmarking Offline Reinforcement Learning on Real-Robot Hardware Gürtler, N., Blaes, S., Kolev, P., Widmaier, F., Wüthrich, M., Bauer, S., Schölkopf, B., Martius, G. In Proceedings of the Eleventh International Conference on Learning Representations, The Eleventh International Conference on Learning Representations (ICLR), May 2023 (Published)

Abstract ›

Learning policies from previously recorded data is a promising direction for real-world robotics tasks, as online learning is often infeasible. Dexterous manipulation in particular remains an open problem in its general form. The combination of offline reinforcement learning with large diverse datasets, however, has the potential to lead to a breakthrough in this challenging domain analogously to the rapid progress made in supervised learning in recent years. To coordinate the efforts of the research community toward tackling this problem, we propose a benchmark including: i) a large collection of data for offline learning from a dexterous manipulation platform on two tasks, obtained with capable RL agents trained in simulation; ii) the option to execute learned policies on a real-world robotic system and a simulation for efficient debugging. We evaluate prominent open-sourced offline reinforcement learning algorithms on the datasets and provide a reproducible experimental setup for offline reinforcement learning on real systems.

Website arXiv Code URL BibTeX

Autonomous Learning Empirical Inference Conference Paper DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems Schumacher, P., Haeufle, D. F., Büchler, D., Schmitt, S., Martius, G. In The Eleventh International Conference on Learning Representations (ICLR), May 2023 (Published)

Abstract ›

Muscle-actuated organisms are capable of learning an unparalleled diversity of dexterous movements despite their vast amount of muscles. Reinforcement learning (RL) on large musculoskeletal models, however, has not been able to show similar performance. We conjecture that ineffective exploration in large overactuated action spaces is a key problem. This is supported by our finding that common exploration noise strategies are inadequate in synthetic examples of overactuated systems. We identify differential extrinsic plasticity (DEP), a method from the domain of self-organization, as being able to induce state-space covering exploration within seconds of interaction. By integrating DEP into RL, we achieve fast learning of reaching and locomotion in musculoskeletal systems, outperforming current approaches in all considered tasks in sample efficiency and robustness.

Arxiv pdf Website URL BibTeX

Autonomous Learning Empirical Inference Conference Paper Bridging the Gap to Real-World Object-Centric Learning Seitzer, M., Horn, M., Zadaianchuk, A., Zietlow, D., Xiao, T., Simon-Gabriel, C., He, T., Zhang, Z., Schölkopf, B., Brox, T., Locatello, F. In Proceedings of the Eleventh International Conference on Learning Representations, The Eleventh International Conference on Learning Representations (ICLR), May 2023 (Published)

Abstract ›

Humans naturally decompose their environment into entities at the appropriate level of abstraction to act in the world. Allowing machine learning algorithms to derive this decomposition in an unsupervised way has become an important line of research. However, current methods are restricted to simulated data or require additional information in the form of motion or depth in order to successfully discover objects. In this work, we overcome this limitation by showing that reconstructing features from models trained in a self-supervised manner is a sufficient training signal for object-centric representations to arise in a fully unsupervised way. Our approach, DINOSAUR, significantly out-performs existing object-centric learning models on simulated data and is the first unsupervised object-centric model that scales to real world-datasets such as COCO and PASCAL VOC. DINOSAUR is conceptually simple and shows competitive performance compared to more involved pipelines from the computer vision literature.

Code Website URL BibTeX

Empirical Inference Conference Paper Effective Bayesian Heteroscedastic Regression with Deep Neural Networks Immer, A. P. E. M. A. V. J. E. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 1-24, Curran Associates Inc., NeurIPS, 2023 (Published) DOI URL BibTeX

Human Aspects of Machine Learning Empirical Inference Conference Paper Robustness implies fairness in causal algorithmic recourse. Ehyaei, A., Karimi, A., Schölkopf, B., Maghsudi, S. In 2023 (Published) BibTeX

Empirical Inference Conference Paper A Hierarchical Model of Attention over Time Kim, J., Singh, S., Yurovsky, D., Fisher, A. A. E. In Proceedings of the 44th Annual Meeting of the Cognitive Science Society (Cogsci 2022), 350-357 , (Editors: J. Culbertson and A. Perfors and H. Rabagliati and V. Ramenzoni), 44st Annual Meeting of the Cognitive Science Society (CogSci 2022) , July 2022 (Published) URL BibTeX

Empirical Inference Conference Paper Kernel interpolation in Sobolev spaces is not consistent in low dimensions Buchholz, S. In Proceedings of 35th Conference on Learning Theory (COLT), 178:3410-3440, Proceedings of Machine Learning Research, (Editors: Loh, Po-Ling and Raginsky, Maxim), PMLR, July 2022 (Published) URL BibTeX

Empirical Inference Conference Paper Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance Ni, J., Jin, Z., Freitag, M., Sachan, M., Schölkopf, B. In Proceedings of the 2022 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 5303-5320, (Editors: Marine Carpuat and Marie-Catherine de Marneffe and Iván Vladimir Meza Ruı́z), Association for Computational Linguistics, July 2022 (Published) arXiv DOI URL BibTeX

Probabilistic Learning Group Empirical Inference Conference Paper Don’t Throw it Away! The Utility of Unlabeled Data in Fair Decision Making Rateike, M., Majumdar, A., Mineeva, O., Gummadi, K. P., Valera, I. In FAccT ’22: 2022 ACM Conference on Fairness, Accountability, and Transparency, 1421-1433, ACM, New York, NY, 5th ACM Conference on Fairness, Accountability, and Transparency (FAccT 2022), June 2022 (Published) DOI BibTeX

Statistical Learning Theory Empirical Inference Conference Paper A Bandit Model for Human-Machine Decision Making with Private Information and Opacity Bordt, S., von Luxburg, U. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS), 151:7300-7319, Proceedings of Machine Learning Research, (Editors: Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel), PMLR, March 2022 (Published) URL BibTeX

Empirical Inference Conference Paper Real Robot Challenge: A Robotics Competition in the Cloud Bauer, S., Wüthrich, M., Widmaier, F., Buchholz, A., Stark, S., Goyal, A., Steinbrenner, T., Akpo, J., Joshi, S., Berenz, V., Agrawal, V., Funk, N., Urain, J., Peters, J., Watson, J., Chen, C., Srinivasan, K., Zhang, J., Zhang, J., Walter, M., et al. In Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, 176:190-204, Proceedings of Machine Learning Research, (Editors: Kiela, Douwe and Ciccone, Marco and Caputo, Barbara), PMLR, NeurIPS 2021 Competitions and Demonstrations Track (NeurIPS 2021), 2022 (Published)

Abstract ›

Dexterous manipulation remains an open problem in robotics. To coordinate efforts of the research community towards tackling this problem, we propose a shared benchmark. We designed and built robotic platforms that are hosted at the MPI-IS and can be accessed remotely. Each platform consists of three robotic fingers that are capable of dexterous object manipulation. Users are able to control the platforms remotely by submitting code that is executed automatically, akin to a computational cluster. Using this setup, i) we host robotics competitions, where teams from anywhere in the world access our platforms to tackle challenging tasks ii) we publish the datasets collected during these competitions (consisting of hundreds of robot hours), and iii) we give researchers access to these platforms for their own projects.

URL BibTeX

Autonomous Learning Empirical Inference Conference Paper Causal Influence Detection for Improving Efficiency in Reinforcement Learning Seitzer, M., Schölkopf, B., Martius, G. In Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 34:22905-22918, (Editors: M. Ranzato and A. Beygelzimer and Y. Dauphin and P. S. Liang and J. Wortman Vaughan), Curran Associates, Inc., Red Hook, NY, 35th Conference on Neural Information Processing Systems, December 2021 (Published) arXiv PDF Data Code URL BibTeX

Empirical Inference Conference Paper DiBS: Differentiable Bayesian Structure Learning Lorch, L., Rothfuss, J., Schölkopf, B., Krause, A. In Advances in Neural Information Processing Systems 34, 29:24111-24123, (Editors: M. Ranzato and A. Beygelzimer and Y. Dauphin and P. S. Liang and J. Wortman Vaughan), Curran Associates, Inc., Red Hook, NY, 35th Annual Conference on Neural Information Processing Systems (NeurIPS 2021), December 2021 (Published) URL BibTeX

Empirical Inference Autonomous Learning Conference Paper Hierarchical Reinforcement Learning with Timed Subgoals Gürtler, N., Büchler, D., Martius, G. In Advances in Neural Information Processing Systems 34, 26:21732-21743, (Editors: M. Ranzato and A. Beygelzimer and Y. Dauphin and P. S. Liang and J. Wortman Vaughan), Curran Associates, Inc., Red Hook, NY, 35th Conference on Neural Information Processing Systems (NeurIPS 2021), December 2021 (Published) video arXiv code URL BibTeX

Empirical Inference Conference Paper Bayesian Deep Learning via Subnetwork Inference Daxberger, E., Nalisnick, E., Allingham, J., Antorán, J., Hernández-Lobato, J. M. In Proceedings of 38th International Conference on Machine Learning (ICML), 139:2510-2521, Proceedings of Machine Learning Research, (Editors: Meila, Marina and Zhang, Tong), PMLR, 38th International Conference on Machine Learning (ICML 2021), July 2021 (Published) URL BibTeX

Empirical Inference Conference Paper Orthogonal Over-Parameterized Training Liu, W., Lin, R., Liu, Z., Rehg, J., Paull, L., Xiong, L., Song, L., Weller, A. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 7251-7260, Computer Vision Foundation / IEEE, CVPR, June 2021 (Published) URL BibTeX

Empirical Inference Conference Paper CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning Ahmed*, O., Träuble*, F., Goyal, A., Neitz, A., Bengio, Y., Schölkopf, B., Wüthrich, M., Bauer, S. In 9th International Conference on Learning Representations (ICLR 2021), 20, ICLR, Wien, International Conference on Learning Representations (ICLR), May 2021, *equal contribution (Published) URL BibTeX

Empirical Inference Conference Paper A teacher-student framework to distill future trajectories Neitz*, A., Parascandolo*, G., Schölkopf, B. In 9th International Conference on Learning Representations (ICLR), May 2021, *equal contribution (Published) URL BibTeX

Empirical Inference Conference Paper Fast And Slow Learning Of Recurrent Independent Mechanisms Madan, K., Ke, N. R., Goyal, A., Schölkopf, B., Bengio, Y. In 9th International Conference on Learning Representations (ICLR), May 2021 (Published) URL BibTeX

Empirical Inference Conference Paper Learning explanations that are hard to vary Parascandolo*, G., Neitz*, A., Orvieto, A., Gresele, L., Schölkopf, B. In 9th International Conference on Learning Representations (ICLR), May 2021, *equal contribution (Published) arXiv URL BibTeX

Empirical Inference Conference Paper Predicting Infectiousness for Proactive Contact Tracing Bengio, Y., Gupta, P., Maharaj, T., Rahaman, N., Weiss, M., Deleu, T., Muller, E. B., Qu, M., Schmidt, V., St-Charles, P., Alsdurf, H., Bilaniuk, O., Buckeridge, D., Marceau-Caron, G., Carrier, P., Ghosn, J., Ortiz Gagne, S., Pal, C., Rish, I., Schölkopf, B., et al. In The Ninth International Conference on Learning Representations (ICLR 2021), 9th International Conference on Learning Representations (ICLR), May 2021 (Published) URL BibTeX

Empirical Inference Conference Paper Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator Paulus, M. B., Maddison, C. J., Krause, A. In The Ninth International Conference on Learning Representations , 9th International Conference on Learning Representations (ICLR 2021), May 2021 (Published) URL BibTeX

Empirical Inference Conference Paper Spatially Structured Recurrent Modules Rahaman, N., Goyal, A., Gondal, M. W., Wüthrich, M., Bauer, S., Sharma, Y., Bengio, Y., Schölkopf, B. In 9th International Conference on Learning Representations (ICLR), May 2021 (Published) URL BibTeX

Empirical Inference Conference Paper On the Transfer of Disentangled Representations in Realistic Settings Dittadi*, A., Träuble*, F., Locatello, F., Wüthrich, M., Agrawal, V., Winther, O., Bauer, S., Schölkopf, B. In The Ninth International Conference on Learning Representations (ICLR), The 9th International Conference on Learning Representations (ICLR 2021) , May 2021, *equal contribution (Published) URL BibTeX

Empirical Inference Conference Paper Recurrent Independent Mechanisms Goyal, A., Lamb, A., Hoffmann, J., Sodhani, S., Levine, S., Bengio, Y., Schölkopf, B. In The Ninth International Conference on Learning Representations (ICLR), 9th International Conference on Learning Representations (ICLR 2021), May 2021 (Published) URL BibTeX

Empirical Inference Conference Paper ResNet After All: Neural ODEs and Their Numerical Solution Ott, K., Katiyar, P., Hennig, P., Tiemann, M. In The Ninth International Conference on Learning Representations (ICLR 2021), 9th International Conference on Learning Representations (ICLR), May 2021 (Published) URL BibTeX

Empirical Inference Conference Paper Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling Miladinović, D., Stanić, A., Bauer, S., Schmidhuber, J., Buhmann, J. M. In The 9th International Conference on Learning Representations (ICLR), The Ninth International Conference on Learning Representations (ICLR 2021), May 2021 (Published) URL BibTeX

Empirical Inference Conference Paper A Theory of Independent Mechanisms for Extrapolation in Generative Models Besserve, M., Sun, R., Janzing, D., Schölkopf, B. In Proceedings of the 35th AAAI Conference on Artificial Intelligence , 35(8):6741-6749, 35th AAAI Conference on Artificial Intelligence (AAAI 2021), February 2021 (Published) arXiv DOI URL BibTeX

Publications

Filter by