Publications

DEPARTMENTS

Emperical Interference

Haptic Intelligence

Modern Magnetic Systems

Perceiving Systems

Physical Intelligence

Robotic Materials

Social Foundations of Computation


Research Groups

Autonomous Vision

Autonomous Learning

Bioinspired Autonomous Miniature Robots

Dynamic Locomotion

Embodied Vision

Human Aspects of Machine Learning

Intelligent Control Systems

Learning and Dynamical Systems

Locomotion in Biorobotic and Somatic Systems

Micro, Nano, and Molecular Systems

Movement Generation and Control

Neural Capture and Synthesis

Physics for Inference and Optimization

Organizational Leadership and Diversity

Probabilistic Learning Group


Topics

Robot Learning

Conference Paper

2022

Autonomous Learning

Robotics

AI

Career

Award


Rationality Enhancement Article Learning to Overexert Cognitive Control in a Stroop Task Bustamante, L., Lieder, F., Musslick, S., Shenhav, A., Cohen, J. Cognitive, Affective, & Behavioral Neuroscience, 21:453-471, January 2021, Laura Bustamante and Falk Lieder contributed equally to this publication. (Published)
How do people learn when to allocate how much cognitive control to which task? According to the Learned Value of Control (LVOC) model, people learn to predict the value of alternative control allocations from features of a given situation. This suggests that people may generalize the value of control learned in one situation to other situations with shared features, even when the demands for cognitive control are different. This makes the intriguing prediction that what a person learned in one setting could, under some circumstances, cause them to misestimate the need for, and potentially over-exert control in another setting, even if this harms their performance. To test this prediction, we had participants perform a novel variant of the Stroop task in which, on each trial, they could choose to either name the color (more control-demanding) or read the word (more automatic). However only one of these tasks was rewarded, it changed from trial to trial, and could be predicted by one or more of the stimulus features (the color and/or the word). Participants first learned colors that predicted the rewarded task. Then they learned words that predicted the rewarded task. In the third part of the experiment, we tested how these learned feature associations transferred to novel stimuli with some overlapping features. The stimulus-task-reward associations were designed so that for certain combinations of stimuli the transfer of learned feature associations would incorrectly predict that more highly rewarded task would be color naming, which would require the exertion of control, even though the actually rewarded task was word reading and therefore did not require the engagement of control. Our results demonstrated that participants over-exerted control for these stimuli, providing support for the feature-based learning mechanism described by the LVOC model.
Learning to Overexert Cognitive Control in a Stroop Task Learning to Overexert Cognitive Control in a Stroop Tas DOI URL BibTeX

Movement Generation and Control Conference Paper Meta-Learning via Learned Loss Bechtle, S., Molchanov, A., Chebotar, Y., Grefenstette, E., Righetti, L., Sukhatme, G., Meier, F. In 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, January 2021 (Published)
Typically, loss functions, regularization mechanisms and other important aspects of training parametric models are chosen heuristically from a limited set of options. In this paper, we take the first step towards automating this process, with the view of producing models which train faster and more robustly. Concretely, we present a meta-learning method for learning parametric loss functions that can generalize across different tasks and model architectures. We develop a pipeline for “meta-training” such loss functions, targeted at maximizing the performance of the model trained under them. The loss landscape produced by our learned losses significantly improves upon the original task-specific losses in both supervised and reinforcement learning tasks. Furthermore, we show that our meta-learning framework is flexible enough to incorporate additional informa- tion at meta-train time. This information shapes the learned loss function such that the environment does not need to provide this information during meta-test time. We make our code available at https://sites.google.com/view/mlthree
BibTeX

Autonomous Learning Haptic Intelligence Patent Method for force inference, method for training a feed-forward neural network, force inference module, and sensor arrangement Sun, H., Martius, G., Kuchenbecker, K. J. (PCT/EP2021/050231), Max Planck Institute for Intelligent Systems, Max Planck Ring 4, January 2021
The invention relates to a method for force inference of a sensor arrangement for sensing forces, to a method for training a feed-forward neural network, to a force inference module, and to a sensor arrangement.
BibTeX

Empirical Inference Conference Paper Neural Linear Models with Functional Gaussian Process Priors Watson, J., Lin, J. A., Klink, P., Peters, J. 3rd Symposium on Advances in Approximate Bayesian Inference , 3rd Symposium on Advances in Approximate Bayesian Inference (AABI 2021) , January 2021 (Published) URL BibTeX

Movement Generation and Control Article Reactive Balance Control for Legged Robots under Visco-Elastic Contacts Flayols, T., Prete, A. D., Khadiv, M., Mansard, N., Righetti, L. Applied Sciences, 11(1), MDPI, January 2021 (Published)
Contacts between robots and environment are often assumed to be rigid for control purposes. This assumption can lead to poor performance when contacts are soft and/or underdamped. However, the problem of balancing on soft contacts has not received much attention in the literature. This paper presents two novel approaches to control a legged robot balancing on visco-elastic contacts, and compares them to other two state-of-the-art methods. Our simulation results show that performance heavily depends on the contact stiffness and the noises/uncertainties introduced in the simulation. Briefly, the two novel controllers performed best for soft/medium contacts, whereas “inverse-dynamics control under rigid-contact assumptions” was the best one for stiff contacts. Admittance control was instead the most robust, but suffered in terms of performance. These results shed light on this challenging problem, while pointing out interesting directions for future investigation.
DOI URL BibTeX

Autonomous Learning Haptic Intelligence Patent Sensor Arrangement for Sensing Forces and Methods for Fabricating a Sensor Arrangement and Parts Thereof Sun, H., Martius, G., Kuchenbecker, K. J. (PCT/EP2021/050230), Max Planck Institute for Intelligent Systems, Max Planck Ring 4, January 2021
The invention relates to a vision-based haptic sensor arrangement for sensing forces, to a method for fabricating a top portion of a sensor arrangement, and to a method for fabricating a sensor arrangement.
BibTeX

Perceiving Systems Conference Paper SMPLpix: Neural Avatars from 3D Human Models Prokudin, S., Black, M. J., Romero, J. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV 2021), 1809-1818, IEEE, Piscataway, NJ, IEEE Winter Conference on Applications of Computer Vision (WACV 2021), January 2021 (Published)
Recent advances in deep generative models have led to an unprecedented level of realism for synthetically generated images of humans. However, one of the remaining fundamental limitations of these models is the ability to flexibly control the generative process, e.g. change the camera and human pose while retaining the subject identity. At the same time, deformable human body models like SMPL \cite{loper2015smpl} and its successors provide full control over pose and shape, but rely on classic computer graphics pipelines for rendering. Such rendering pipelines require explicit mesh rasterization that (a) does not have the potential to fix artifacts or lack of realism in the original 3D geometry and (b) until recently, were not fully incorporated into deep learning frameworks. In this work, we propose to bridge the gap between classic geometry-based rendering and the latest generative networks operating in pixel space. We train a network that directly converts a sparse set of 3D mesh vertices into photorealistic images, alleviating the need for traditional rasterization mechanism. We train our model on a large corpus of human 3D models and corresponding real photos, and show the advantage over conventional differentiable renderers both in terms of the level of photorealism and rendering efficiency.
project official pdf video preprint code DOI BibTeX

Physical Intelligence Article 4D printing of continuous shape representation Ahn, S., Byun, J., Joo, H., Jeong, J., Lee, D., Cho, K. Advanced Materials Technologies, 6(6):2100133, 2021
Abstract 4D printing can address time-evolving structural functions that are unattainable by conventional 3D printing. Despite the advance in materials and printing techniques, however, 4D printing of continuity of shape representation that generally characterizes 3D matters is still challenging, because the existing methodologies mostly rely on a few discrete levels of strain and their spatial distributions. Here, a 4D printing strategy of shape memory polymers (SMPs) that can program continuous levels of shape-recovery strain is proposed. It is found that the irrecoverable state of the SMP and the corresponding recovery strain can be controlled in a continuous and precise manner by a single printing parameter. Importantly, the continuity of strain programming provides an opportunity for the translation into mathematical function representation (F-rep), which allows the systematic derivation and implementation of 4D-printed bilayer strain functions that are matched to the continuously varying curvatures of the target geometry. Combined with the custom-built software, the F-rep 4D printing strategy can produce 4D-printed architectures that involve continuously varying strain profiles of almost any function type. The effectiveness of the framework is highlighted by a set of 3D face masks with facial feature transformation driven by a function operator.
DOI URL BibTeX

Autonomous Learning Article A Reinforcement Learning Approach to View Planning for Automated Inspection Tasks Landgraf, C., Meese, B., Pabst, M., Martius, G., Huber, M. F. Sensors, 21(6):2030, MDPI, 2021 (Published) DOI BibTeX

Empirical Inference Robotics Miscellaneous A Robot Cluster for Reproducible Research in Dexterous Manipulation Wüthrich*, M., Widmaier*, F., Bauer*, S., Funk, N., Urain, J., Peters, J., Watson, J., Chen, C., Srinivasan, K., Zhang, J., Zhang, J., Walter, M. R., Madan, R., Schaff, C., Maeda, T., Yoneda, T., Yarats, D., Allshire, A., Gordon, E. K., Bhattacharjee, T., et al. 2021, *equal contribution (Published) arXiv BibTeX

Autonomous Motion Conference Paper A System for Traded Control Teleoperation of Manipulation Tasks using Intent Prediction from Hand Gestures Oh, Y., Schäfer, T., Rüther, B., Toussaint, M., Mainprice, J. In 2021 30th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2021), 503-508, IEEE, Piscataway, NJ, 30th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2021), 2021 (Published) DOI BibTeX

Robotic Materials Article A Triboelectric-Nanogenerator-Based Gas–Solid Two-Phase Flow Sensor for Pneumatic Conveying System Detecting Wang, Y., Liu, D., Hu, Z., Chen, T., Zhang, Z., Wang, H., Du, T., Zhang, S. L., Zhao, Z., Zhou, T., Xu, M. Advanced Materials Technologies, 6(5):2001270, 2021 (Published) DOI BibTeX

Article A diffusion wavelets-based multiscale framework for inverse optimal control of stochastic systems Ha, J., Chae, H., Choi, H. International Journal of Systems Science, 52(11):2228-2240, 2021 (Published) DOI BibTeX

Software Workshop Article A pupillary index of susceptibility to decision biases Eldar, E., Felso, V., Cohen, J. D., Niv, Y. Nature Human Behaviour, 5(5):653-662, 2021 (Published) DOI BibTeX

Conference Paper ACTrain@School: Can we bring AI to the classroom to foster self-regulated learning? Wirzberger, M. EARLI, 2021
Learning requires a multitude of metacognitive activities that support knowledge acquisition and direct the learning process. These include selecting and planning goals, applying, observing and evaluating learning strategies as well as regulatory efforts required for goal achievement. We introduce an example of how learners can be supported to set goals, implement periods of focused work and integrate a meaningful break management by using the AI-based training software ACTrain. Metacognitive feedback based on principles of machine learning conveys the value of staying focused on goal-related activities. Pilot results already indicate positive effects of the presented approach and suggest further investigation. We will explore potentials for integrating such approach in school settings and how this could affect the role of teachers.
URL BibTeX

Theory of Inhomogeneous Condensed Matter Article Active spheres induce Marangoni flows that drive collective dynamics Wittmann, M., Popescu, M. N., Dominguez, A., Simmchen, J. The European Physical Journal E, 44(2):15, EDP Sciences; Societá Italiana di Fisica; Springer, Les Ulis; Bologna; Heidelberg, 2021 (Published) DOI BibTeX

Theory of Inhomogeneous Condensed Matter Article Activity-Induced Collapse and Arrest of Active Polymer Rings Locatelli, E., Bianco, V., Malgaretti, P. Physical Review Letters, 126(9):097801, American Physical Society, Woodbury, N.Y., 2021 (Published) DOI BibTeX

Perceiving Systems Article Analyzing the Direction of Emotional Influence in Nonverbal Dyadic Communication: A Facial-Expression Study Shadaydeh, M., Müller, L., Schneider, D., Thuemmel, M., Kessler, T., Denzler, J. IEEE Access, 9:73780-73790, IEEE, 2021 (Published)
Identifying the direction of emotional influence in a dyadic dialogue is of increasing interest in the psychological sciences with applications in psychotherapy, analysis of political interactions, or interpersonal conflict behavior. Facial expressions are widely described as being automatic and thus hard to be overtly influenced. As such, they are a perfect measure for a better understanding of unintentional behavior cues about socio-emotional cognitive processes. With this view, this study is concerned with the analysis of the direction of emotional influence in dyadic dialogues based on facial expressions only. We exploit computer vision capabilities along with causal inference theory for quantitative verification of hypotheses on the direction of emotional influence, i.e., cause-effect relationships, in dyadic dialogues. We address two main issues. First, in a dyadic dialogue, emotional influence occurs over transient time intervals and with intensity and direction that are variant over time. To this end, we propose a relevant interval selection approach that we use prior to causal inference to identify those transient intervals where causal inference should be applied. Second, we propose to use fine-grained facial expressions that are present when strong distinct facial emotions are not visible. To specify the direction of influence, we apply the concept of Granger causality to the time-series of facial expressions over selected relevant intervals. We tested our approach on newly, experimentally obtained data. Based on quantitative verification of hypotheses on the direction of emotional influence, we were able to show that the proposed approach is promising to reveal the cause-effect pattern in various instructed interaction conditions.
DOI BibTeX

Theory of Inhomogeneous Condensed Matter Article Antiresonant driven systems for particle manipulation Carusela, M. F., Malgaretti, P., Rubi, J. M. Physical Review E, 103(6):062102, American Physical Society, Melville, NY, 2021 DOI BibTeX

Empirical Inference Article Are intrinsic neural timescales related to sensory processing? Evidence from abnormal behavioral states Zilio, F., Gomez-Pilar, J., Cao, S., Zhang, J., Zang, D., Qi, Z., Tan, J., Hiromi, T., Wu, X., Fogel, S., Huang, Z., Hohmann, M. R., Fomina, T., Synofzik, M., Grosse-Wentrup, M., Owen, A. M., Northoff, G. NeuroImage, 226:14, 2021 (Published) DOI URL BibTeX

Empirical Inference Article Assessing Transferability From Simulation to Reality for Reinforcement Learning Muratore, F., Gienger, M., Peters, J. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(4):1172-1183, 2021 (Published) DOI URL BibTeX

Physical Intelligence Modern Magnetic Systems Article Bayesian Machine Learning for Efficient Minimization of Defects in ALD Passivation Layers Dogan, G., Demir, S. O., Gutzler, R., Gruhn, H., Dayan, C. B., Sanli, U. T., Silber, C., Culha, U., Sitti, M., Schütz, G., Grévent, C., Keskinbora, K. ACS Applied Materials and Interfaces, 13(45):54503-54515, American Chemical Society, Washington, DC, 2021 DOI BibTeX

Empirical Inference Article Bayesian ODE solvers: the maximum a posteriori estimate Tronarp, F., Särkkä, S., Hennig, P. Statistics and Computing, 31(3):23, 2021 (Published) DOI BibTeX

Theory of Inhomogeneous Condensed Matter Article Behavior-dependent critical dynamics in collective states of active particles Loeffler, R. C., Bäuerle, T., Kardar, M., Rohwer, C. M., Bechinger, C. EPL (Europhysics Letters), 134(6):64001, IOP Publishing, Bristol, 2021 DOI BibTeX

Autonomous Vision Article Benchmarking Unsupervised Object Representations for Video Sequences Weis, M., Chitta, K., Sharma, Y., Brendel, W., Bethge, M., Geiger, A., Ecker, A. Journal of Machine Learning Research (JMLR), 22:61, 2021 (Published)
Perceiving the world in terms of objects and tracking them through time is a crucial prerequisite for reasoning and scene understanding. Recently, several methods have been proposed for unsupervised learning of object-centric representations. However, since these models were evaluated on different downstream tasks, it remains unclear how they compare in terms of basic perceptual abilities such as detection, figure-ground segmentation and tracking of objects. To close this gap, we design a benchmark with four data sets of varying complexity and seven additional test sets featuring challenging tracking scenarios relevant for natural videos. Using this benchmark, we compare the perceptual abilities of four object-centric approaches: ViMON, a video-extension of MONet, based on recurrent spatial attention, OP3, which exploits clustering via spatial mixture models, as well as TBA and SCALOR, which use explicit factorization via spatial transformers. Our results suggest that the architectures with unconstrained latent representations learn more powerful representations in terms of object detection, segmentation and tracking than the spatial transformer based architectures. We also observe that none of the methods are able to gracefully handle the most challenging tracking scenarios despite their synthetic nature, suggesting that our benchmark may provide fruitful guidance towards learning more robust object-centric video representations.
Paper Project page URL BibTeX

Perceiving Systems Article Body Image Disturbances and Weight Bias After Obesity Surgery: Semantic and Visual Evaluation in a Controlled Study, Findings from the BodyTalk Project Meneguzzo, P., Behrens, S. C., Favaro, A., Tenconi, E., Vindigni, V., Teufel, M., Skoda, E., Lindner, M., Quiros-Ramirez, M. A., Mohler, B., Black, M., Zipfel, S., Giel, K. E., Pavan, C. Obesity Surgery, 31(4):1625-1634, 2021 (Published)
Purpose: Body image has a significant impact on the outcome of obesity surgery. This study aims to perform a semantic evaluation of body shapes in obesity surgery patients and a group of controls. Materials and Methods: Thirty-four obesity surgery (OS) subjects, stable after weight loss (average 48.03 ± 18.60 kg), and 35 overweight/obese controls (MC), were enrolled in this study. Body dissatisfaction, self-esteem, and body perception were evaluated with self-reported tests, and semantic evaluation of body shapes was performed with three specific tasks constructed with realistic human body stimuli. Results: The OS showed a more positive body image compared to HC (p < 0.001), higher levels of depression (p < 0.019), and lower self-esteem (p < 0.000). OS patients and HC showed no difference in weight bias, but OS used a higher BMI than HC in the visualization of positive adjectives (p = 0.011). Both groups showed a mental underestimation of their body shapes. Conclusion: OS patients are more psychologically burdened and have more difficulties in judging their bodies than overweight/obese peers. Their mental body representations seem not to be linked to their own BMI. Our findings provide helpful insight for the design of specific interventions in body image in obese and overweight people, as well as in OS.
paper pdf DOI BibTeX

Theory of Inhomogeneous Condensed Matter Article Brownian systems perturbed by mild shear: comparing response relations Asheichyk, K., Fuchs, M., Krüger, M. Journal of Physics: Condensed Matter, 33(40):405101, IOP Publishing, Bristol, 2021 DOI BibTeX

Theory of Inhomogeneous Condensed Matter Article Capacitive energy storage in single-file pores: Exactly solvable models and simulations Verkholyak, T., Kuzmak, A., Kondrat, S. The Journal of Chemical Physics, 155(17):174112, American Institute of Physics, Woodbury, N.Y., 2021 DOI BibTeX

Theory of Inhomogeneous Condensed Matter Article Capillary Ionization and Jumps of Capacitive Energy Stored in Mesopores Cruz, C., Kondrat, S., Lomba, E., Ciach, A. The Journal of Physical Chemistry C, 125(19):10243-10249, American Chemical Society, Washington, D.C., 2021 DOI BibTeX

Conference Paper Co-Optimizing Robot, Environment, and Tool Design via Joint Manipulation Planning Toussaint, M., Ha, J., Oguz, O. S. In 2021 IEEE International Conference on Robotics and Automation (ICRA 2021), 6600-6606, IEEE, Piscataway, NJ, IEEE International Conference on Robotics and Automation (ICRA 2021), 2021 (Published) DOI BibTeX

Perceiving Systems Conference Paper Collaborative Mapping of Archaeological Sites Using Multiple UAVs Patel, M., Bandopadhyay, A., Ahmad, A. In Lecture Notes in Networks and Systems, 412:54-70, Springer International Publishing, Intelligent Autonomous Systems 16, 2021
UAVs have found an important application in archaeological mapping. Majority of the existing methods employ an offline method to process the data collected from an archaeological site. They are time-consuming and computationally expensive. In this paper, we present a multi-UAV approach for faster mapping of archaeological sites. Employing a team of UAVs not only reduces the mapping time by distribution of coverage area, but also improves the map accuracy by exchange of information. Through extensive experiments in a realistic simulation (AirSim), we demonstrate the advantages of using a collaborative mapping approach. We then create the first 3D map of the Sadra Fort, a 15th Century Fort located in Gujarat, India using our proposed method. Additionally, we present two novel archaeological datasets recorded in both simulation and real-world to facilitate research on collaborative archaeological mapping. For the benefit of the community, we make the AirSim simulation environment, as well as the datasets publicly available (Project web page: http://patelmanthan.in/castle-ruins-airsim/).
DOI BibTeX

Modern Magnetic Systems Article Competing spin wave emission mechanisms revealed by time-resolved x-ray microscopy Träger, N., Lisiecki, F., Lawitzki, R., Weigand, M., Glowinski, H., Schütz, G., Schmitz, G., Kuswik, P., Krawczyk, M., Gräfe, J., Gruszecki, P. Physical Review B, 103(1):014430, American Physical Society, Woodbury, NY, 2021 DOI BibTeX

Theory of Inhomogeneous Condensed Matter Article Conformation-changing enzymes and macromolecular crowding Skóra, T., Popescu, M. N., Kondrat, S. Physical Chemistry Chemical Physics, 23(15):9065-9069, Royal Society of Chemistry, Cambridge, England, 2021 DOI BibTeX

Statistical Learning Theory Article Contextual Cueing May Not Be Unconscious Meyen, S. V. L. U. F. V. H. Perception, 50(1_Suppl):51-51, 2021 (Published) BibTeX

Theory of Inhomogeneous Condensed Matter Article Continuous nonequilibrium transition driven by heat flow Zhang, Y., Litniewski, M., Makuch, K., Zuk, P. J., Maciolek, A., Holyst, R. Physical Review E, 104(2):024102, American Physical Society, Melville, NY, 2021 DOI BibTeX

Theory of Inhomogeneous Condensed Matter Article Controlled deposition of nanoparticles with critical Casimir forces Marino, E., Vasilyev, O. A., Kluft, B. B., Stroink, M. J. B., Kondrat, S., Schall, P. Nanoscale Horizons, 6(9):751-758, Royal Society of Chemistry, Cambridge, England, 2021 DOI BibTeX

Empirical Inference Article Convolutional neural network-assisted recognition of nanoscale L12 ordered structures in face-centred cubic alloys Li, Y., Zhou, X., Colnaghi, T., Wei, Y., Marek, A., Li, H., Bauer, S., Rampp, M., Stephenson, L. npj Computational Materials, 7, 2021 (Published) PDF DOI URL BibTeX