DEPARTMENTS

Emperical Interference

Haptic Intelligence

Modern Magnetic Systems

Perceiving Systems

Physical Intelligence

Robotic Materials

Social Foundations of Computation


Research Groups

Autonomous Vision

Autonomous Learning

Bioinspired Autonomous Miniature Robots

Dynamic Locomotion

Embodied Vision

Human Aspects of Machine Learning

Intelligent Control Systems

Learning and Dynamical Systems

Locomotion in Biorobotic and Somatic Systems

Micro, Nano, and Molecular Systems

Movement Generation and Control

Neural Capture and Synthesis

Physics for Inference and Optimization

Organizational Leadership and Diversity

Probabilistic Learning Group


Topics

Robot Learning

Conference Paper

2022

Autonomous Learning

Robotics

AI

Career

Award


Haptic Intelligence Optics and Sensing Laboratory Miscellaneous Open-Source Multi-Viewpoint Surgical Telerobotics Caccianiga, G., Sharon, Y., Javot, B., Polikovsky, S., Ergün, G., Capobianco, I., Mihaljevic, A. L., Deguet, A., Kuchenbecker, K. J. Extended abstract (2 pages) presented at the ICRA Workshop on Robot-Assisted Medical Imaging (ICRA-RAMI), Atlanta, USA, May 2025 (Published) URL BibTeX

Haptic Intelligence Empirical Inference Optics and Sensing Laboratory Software Workshop Article Fiber-Optic Shape Sensing Using Neural Networks Operating on Multispecklegrams Cao, C. G. L., Javot, B., Bhattarai, S., Bierig, K., Oreshnikov, I., Volchkov, V. V. IEEE Sensors Journal, 24(17):27532-27540, September 2024 (Published)
Application of machine learning techniques on fiber speckle images to infer fiber deformation allows the use of an unmodified multimode fiber to act as a shape sensor. This approach eliminates the need for complex fiber design or construction (e.g., Bragg gratings and time-of-flight). Prior work in shape determination using neural networks trained on a finite number of possible fiber shapes (formulated as a classification task), or trained on a few continuous degrees of freedom, has been limited to reconstruction of fiber shapes only one bend at a time. Furthermore, generalization to shapes that were not used in training is challenging. Our innovative approach improves generalization capabilities, using computer vision-assisted parameterization of the actual fiber shape to provide a ground truth, and multiple specklegrams per fiber shape obtained by controlling the input field. Results from experimenting with several neural network architectures, shape parameterization, number of inputs, and specklegram resolution show that fiber shapes with multiple bends can be accurately predicted. Our approach is able to generalize to new shapes that were not in the training set. This approach of end-to-end training on parameterized ground truth opens new avenues for fiber-optic sensor applications. We publish the datasets used for training and validation, as well as an out-of-distribution (OOD) test set, and encourage interested readers to access these datasets for their own model development.
DOI BibTeX

Empirical Inference Optics and Sensing Laboratory Conference Paper Polarization-based non-linear deep diffractive neural networks Kottapalli, S. N. M., Schlieder, L., Song, A., Volchkov, V., Schölkopf, B., Fischer, P. AI and Optical Data Sciences V, PC12903:PC129030B, (Editors: Ken-ichi Kitayama and Volker J. Sorger), SPIE, January 2024 (Published) DOI BibTeX

Empirical Inference Optics and Sensing Laboratory Software Workshop Conference Paper Glare Removal for Astronomical Images with High Local Dynamic Range Bastelaer, M., Kremer, H., Volchkov, V., Passy, J., Schölkopf, B. IEEE International Conference on Computational Photography (ICCP), 1-11, IEEE, July 2023 (Published) DOI BibTeX

Perceiving Systems Optics and Sensing Laboratory Conference Paper Capturing and Inferring Dense Full-Body Human-Scene Contact Huang, C. P., Yi, H., Höschle, M., Safroshkin, M., Alexiadis, T., Polikovsky, S., Scharstein, D., Black, M. J. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022), 13264-13275, IEEE, Piscataway, NJ, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022), June 2022 (Published)
Inferring human-scene contact (HSC) is the first step toward understanding how humans interact with their surroundings. While detecting 2D human-object interaction (HOI) and reconstructing 3D human pose and shape (HPS) have enjoyed significant progress, reasoning about 3D human-scene contact from a single image is still challenging. Existing HSC detection methods consider only a few types of predefined contact, often reduce body and scene to a small number of primitives, and even overlook image evidence. To predict human-scene contact from a single image, we address the limitations above from both data and algorithmic perspectives. We capture a new dataset called RICH for “Real scenes, Interaction, Contact and Humans.” RICH contains multiview outdoor/indoor video sequences at 4K resolution, ground-truth 3D human bodies captured using markerless motion capture, 3D body scans, and high resolution 3D scene scans. A key feature of RICH is that it also contains accurate vertex-level contact labels on the body. Using RICH, we train a network that predicts dense body-scene contacts from a single RGB image. Our key insight is that regions in contact are always occluded so the network needs the ability to explore the whole image for evidence. We use a transformer to learn such non-local relationships and propose a new Body-Scene contact TRansfOrmer (BSTRO). Very few methods explore 3D contact; those that do focus on the feet only, detect foot contact as a post-processing step, or infer contact from body pose without looking at the scene. To our knowledge, BSTRO is the first method to directly estimate 3D body-scene contact from a single image. We demonstrate that BSTRO significantly outperforms the prior art. The code and dataset are available at https://rich.is.tue.mpg.de.
project arXiv BSTRO code video DOI URL BibTeX

Optics and Sensing Laboratory Article Bichromatic state-dependent disordered potential for Anderson localization of ultracold atoms Lecoutre, B., Guo, Y., Yu, X., Niranjan, M., Mukhtar, M., Volchkov, V., Aspect, A., Josse, V. European Physical Journal D, 76:218, 2022
The ability to load ultracold atoms at a well-defined energy in a disordered potential is a crucial tool to study quantum transport, and in particular Anderson localization. In this paper, we present a new method for achieving that goal by rf transfer of atoms in an atomic Bose-Einstein condensate from a disorder-insensitive state to a disorder-sensitive state. It is based on a bichromatic laser speckle pattern, produced by two lasers whose frequencies are chosen so that their light-shifts cancel each other in the first state and add up in the second state. Moreover, the spontaneous scattering rate in the disorder-sensitive state is low enough to allow for long observation times of quantum transport in that state. We theoretically and experimentally study the characteristics of the resulting potential.
DOI BibTeX

Optics and Sensing Laboratory Article Invited perspectives: Building sustainable and resilient communities – recommended actions for natural hazard scientists Gill, J. C., Taylor, F. E., Duncan, M. J., Mohadjer, S., Budimir, M., Mdala, H., Bukachi, V. Natural Hazards and Earth System Sciences, 21(1):187-202, 2021 (Published) DOI URL BibTeX

Optics and Sensing Laboratory Article Using paired teaching for earthquake education in schools Mohadjer, S., Mutz, S., Kemp, M., Gill, S., Ischuk, A., Ehlers, T. Geoscience Communication, 4(2):281-295, 2021 (Published)
In this study, we have created 10 geoscience video lessons that follow the paired-teaching pedagogical approach. This method is used to supplement the standard school curriculum with video lessons, instructed by geoscientists from around the world, coupled with activities carried out under the guidance of classroom teachers. The video lessons introduce students to the scientific concepts behind earthquakes (e.g. the Earth’s interior, plate tectonics, faulting, and seismic energy), earthquake hazards, and mitigation measures (e.g. liquefaction, structural, and non- structural earthquake hazards). These concepts are taught through hands-on learning, where students use everyday materials to build models to visualize basic Earth processes that produce earthquakes and explore the effects of different hazards. To evaluate the effectiveness of these virtual lessons, we tested our videos in school classrooms in Dushanbe (Tajikistan) and London (United Kingdom). Before and after the video implementations, students completed questionnaires that probed their knowledge on topics covered by each video, including the Earth’s interior, tectonic plate boundaries, and non-structural hazards. Our assessment results indicate that, while the paired- teaching video lessons appear to enhance student knowledge and understanding of some concepts (e.g. Earth’s interior, earthquake location forecasting, and non-structural hazards), they bring little change to their views on the causes of earthquakes and their relation to plate boundaries. In general, the difference between UK and Tajik students’ level of knowledge prior to and after video testing is more significant than the difference between pre- and post-knowledge for each group. This could be due to several factors affecting curriculum testing (e.g. level of teachers’ participation and classroom culture) and students’ learning of content (e.g. pre-existing hazards knowledge and experience). To maximize the impact of school-based risk reduction education, curriculum developers must move beyond innovative content and pedagogical approaches, take classroom culture into consideration, and instil skills needed for participatory learning and discovery.
DOI URL BibTeX

Optics and Sensing Laboratory Article Temporal variations in rockfall and rock-wall retreat rates in a deglaciated valley over the past 11 k.y. Mohadjer, S., Ehlers, T. A., Nettesheim, M., Ott, M. B., Glotzbach, C., Drews, R. Geology, 48(6):594-598, 2020 (Published) DOI URL BibTeX

Empirical Inference Optics and Sensing Laboratory Autonomous Motion Conference Paper On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset Gondal, M. W., Wüthrich, M., Miladinovic, D., Locatello, F., Breidt, M., Volchkov, V., Akpo, J., Bachem, O., Schölkopf, B., Bauer, S. Advances in Neural Information Processing Systems 32 (NeurIPS 2019), 15714-15725, (Editors: H. Wallach and H. Larochelle and A. Beygelzimer and F. d’Alché-Buc and E. Fox and R. Garnett), Curran Associates, Inc., 33rd Annual Conference on Neural Information Processing Systems, December 2019 (Published) URL BibTeX

Optics and Sensing Laboratory Article Ultracold atoms in disordered potentials: elastic scattering time in the strong scattering regime Signoles, A., Lecoutre, B., Richard, J., Lim, L., Denechaud, V., Volchkov, V., Angelopoulou, V., Jendrzejewski, F., Aspect, A., Sanchez-Palencia, L., Josse, V. New Journal of Physics, 21:105002, IOP Publishing and Deutsche Physikalische Gesellschaft, October 2019 (Published) DOI URL BibTeX

Optics and Sensing Laboratory Article Elastic Scattering Time of Matter-Waves in Disordered Potentials Richard, J., Lim, L., Denechaud, V., Volchkov, V., Lecoutre, B., Mukhtar, M., Jendrzejewski, F., Aspect, A., Signoles, A., Sanchez-Palencia, L., Josse, V. Physical Review Letters, 122:100403, American Physical Society (APS), March 2019 (Published)
We report on an extensive study of the elastic scattering time $τ_\mathrm{s}$ of matter waves in optical disordered potentials. Using direct experimental measurements, numerical simulations, and comparison with the first-order Born approximation based on the knowledge of the disorder properties, we explore the behavior of $τ_\mathrm{s}$ over more than 3 orders of magnitude, ranging from the weak to the strong scattering regime. We study in detail the location of the crossover and, as a main result, we reveal the strong influence of the disorder statistics, especially on the relevance of the widely used Ioffe-Regel-like criterion $k l_\mathrm{s}\sim 1$. While it is found to be relevant for Gaussian-distributed disordered potentials, we observe significant deviations for laser speckle disorders that are commonly used with ultracold atoms. Our results are crucial for connecting experimental investigation of complex transport phenomena, such as Anderson localization, to microscopic theories.
DOI BibTeX

Optics and Sensing Laboratory Article Measurement of Spectral Functions of Ultracold Atoms in Disordered Potentials Volchkov, V., Pasek, M., Denechaud, V., Mukhtar, M., Aspect, A., Delande, D., Josse, V. Physical Review Letters, 120:060404, American Physical Society (APS), January 2018 (Published)
We report on the measurement of the spectral functions of noninteracting ultracold atoms in a three-dimensional disordered potential resulting from an optical speckle field. Varying the disorder strength by 2 orders of magnitude, we observe the crossover from the “quantum” perturbative regime of low disorder to the “classical” regime at higher disorder strength, and find an excellent agreement with numerical simulations. The method relies on the use of state-dependent disorder and the controlled transfer of atoms to create well-defined energy states. This opens new avenues for experimental investigations of three-dimensional Anderson localization.
DOI BibTeX

Optics and Sensing Laboratory Article Little Geodetic Evidence for Localized Indian Subduction in the Pamir-Hindu Kush of Central Asia Perry, M., Kakar, N., Ischuk, A., Metzger, S., Bendick, R., Molnar, P., Mohadjer, S. Geophysical Research Letters, 46(1):109-118, 2018 (Published) DOI URL BibTeX

Optics and Sensing Laboratory Article Seismic monitoring of small alpine rockfalls – validity, precision and limitations Dietze, M., Mohadjer, S., Turowski, J. M., Ehlers, T. A., Hovius, N. Earth Surface Dynamics , 5(4):653–668, 2017 (Published) DOI URL BibTeX

Optics and Sensing Laboratory Article A Quaternary fault database for central Asia Mohadjer, S. E. T. A. B. R. S. K., Strube, T. A. Natural Hazards and Earth System Sciences, 16(16):529-542, 2016 (Published) DOI URL BibTeX