Institute Homepage
Institute Homepage DE Sign In

Publications

DEPARTMENTS

Emperical Interference

Haptic Intelligence

Modern Magnetic Systems

Perceiving Systems

Physical Intelligence

Robotic Materials

Social Foundations of Computation

Research Groups

Autonomous Vision

Autonomous Learning

Bioinspired Autonomous Miniature Robots

Dynamic Locomotion

Embodied Vision

Human Aspects of Machine Learning

Intelligent Control Systems

Learning and Dynamical Systems

Locomotion in Biorobotic and Somatic Systems

Micro, Nano, and Molecular Systems

Movement Generation and Control

Neural Capture and Synthesis

Physics for Inference and Optimization

Organizational Leadership and Diversity

Probabilistic Learning Group

Topics

Robot Learning

Conference Paper

2022

Autonomous Learning

Robotics

AI

Career

Award

Rationality Enhancement Article An interdisciplinary synthesis of research on understanding and promoting well-doing Lieder, F., Prentice, M., Corwin-Renner, E. Social and Personality Psychology Compass, 16(9), September 2022 (Published)
People’s intentional pursuit of prosocial goals and values (i.e., well-doing) is critical to the flourishing of humanity in the long run. Understanding and promoting well-doing is a shared goal across many fields inside and outside of social and personality psychology. Several of these fields are (partially) disconnected from each other and could benefit from more integration of existing knowledge, interdisciplinary collaboration, and cross-fertilization. To foster the transfer and integration of knowledge across these different fields, we provide a brief overview with pointers to some of the key articles in each field, highlight connections, and introduce an integrative model of the psychological mechanisms of well-doing. We identify some gaps in the current understanding of well-doing, such as the paucity of research on well-doing with large and long-lasting positive consequences. Building on this analysis, we identify opportunities for high-impact research on well-doing in social and personality psychology, such as understanding and promoting the effective pursuit of highly impactful altruistic goals.
DOI URL BibTeX
Empirical Inference Conference Paper Any-Shot GIN: Generalizing Implicit Networks for Reconstructing Novel Classes Xian, Y., Chibane, J., Bhatnagar, B. L., Schiele, B., Akata, Z., Pons-Moll, G. International Conference on 3D Vision (3DV), 526-535, September 2022 (Published) DOI URL BibTeX
Empirical Inference Article Causal Feature Selection via Orthogonal Search Soleymani*, A., Raj*, A., Bauer, S., Schölkopf, B., Besserve, M. Transactions on Machine Learning Research, September 2022, *equal contribution (Published) URL BibTeX
Empirical Inference Talk Causality, causal digital twins, and their applications Schölkopf, B. Machine Learning for Science: Bridging Data-Driven and Mechanistic Modelling (Dagstuhl Seminar 22382), (Editors: Berens, Philipp and Cranmer, Kyle and Lawrence, Neil D. and von Luxburg, Ulrike and Montgomery, Jessica), September 2022 (Published) DOI URL BibTeX
Autonomous Learning Conference Paper Developing hierarchical anticipations via neural network-based event segmentation Gumbsch, C., Adam, M., Elsner, B., Martius, G., Butz, M. V. In Proceedings of the IEEE International Conference on Development and Learning (ICDL 2022), 1-8, 2022 IEEE International Conference on Development and Learning (ICDL), September 2022
Humans can make predictions on various time scales and hierarchical levels. Thereby, the learning of event encodings seems to play a crucial role. In this work we model the development of hierarchical predictions via autonomously learned latent event codes. We present a hierarchical recurrent neural network architecture, whose inductive learning biases foster the development of sparsely changing latent state that compress sensorimotor sequences. A higher level network learns to predict the situations in which the latent states tend to change. Using a simulated robotic manipulator, we demonstrate that the system (i) learns latent states that accurately reflect the event structure of the data, (ii) develops meaningful temporal abstract predictions on the higher level, and (iii) generates goal-anticipatory behavior similar to gaze behavior found in eye-tracking studies with infants. The architecture offers a step towards the autonomous learning of compressed hierarchical encodings of gathered experiences and the exploitation of these encodings to generate adaptive behavior.
URL BibTeX
Rationality Enhancement Conference Paper Does deliberate prospection help students set better goals? Jähnichen, S., Weber, F., Prentice, M., Lieder, F. In 15th Biannual Meeting of the German Cognitive Science Society , 188-189 , 15th Biannual Meeting of the German Cognitive Science Society (KogWis 2022 – Understanding Minds) , September 2022 (Published) URL BibTeX
Empirical Inference Article Energy-efficient network activity from disparate circuit parameters Deistler, M., Macke, J. H., Gonçalves, P. J. Proceedings of the National Academy of Sciences, 119(44), September 2022 (Published) DOI BibTeX
Empirical Inference Master Thesis Independent Mechanism Analysis for High Dimensions Sliwa, J. University of Tübingen, Germany, September 2022, (Graduate Training Centre of Neuroscience) (Published) BibTeX
Rationality Enhancement Conference Paper Leveraging AI for effective to-do list gamification Consul, S., Stojcheski, J., Lieder, F. In Mensch und Computer 2022 – Workshopband MuC 2022 , Mensch und Computer 2022 (MuC 2022) : 5th International Workshop "Gam-R – Gamification Reloaded" , September 2022 (Published) DOI URL BibTeX
Physical Intelligence Article Liquid Metal-Elastomer Composites with Dual-Energy Transmission Mode for Multifunctional Miniature Untethered Magnetic Robots Zhang, J., Soon, R. H., Wei, Z., Hu, W., Sitti, M. Advanced science, 9(31):2203730, September 2022 (Published) DOI BibTeX
Empirical Inference Ph.D. Thesis Methods for Minimizing the Spread of Misinformation on the Web Tabibian, B. University of Tübingen, Germany, September 2022 (Published) BibTeX
Empirical Inference Master Thesis Multi-Target Multi-Object Manipulation using Relational Deep Reinforcement Learning Feil, M. Technnical University Munich, Germany, September 2022 (Published) BibTeX
Perceiving Systems Conference Paper Neural Point-based Shape Modeling of Humans in Challenging Clothing Ma, Q., Yang, J., Black, M. J., Tang, S. In 2022 International Conference on 3D Vision (3DV 2022), 679-689, IEEE, Piscataway, NJ, International Conference on 3D Vision (3DV 2022), September 2022 (Published)
Parametric 3D body models like SMPL only represent minimally-clothed people and are hard to extend to cloth- ing because they have a fixed mesh topology and resolution. To address this limitation, recent work uses implicit surfaces or point clouds to model clothed bodies. While not limited by topology, such methods still struggle to model clothing that deviates significantly from the body, such as skirts and dresses. This is because they rely on the body to canonicalize the clothed surface by reposing it to a reference shape. Unfortunately, this process is poorly defined when clothing is far from the body. Additionally, they use linear blend skinning to pose the body and the skinning weights are tied to the underlying body parts. In contrast, we model the clothing deformation in a local coordinate space without canonicalization. We also relax the skinning weights to let multiple body parts influence the surface. Specifically, we extend point-based methods with a coarse stage, that replaces canonicalization with a learned pose- independent “coarse shape” that can capture the rough surface geometry of clothing like skirts. We then refine this using a network that infers the linear blend skinning weights and pose dependent displacements from the coarse representation. The approach works well for garments that both conform to, and deviate from, the body. We demonstrate the usefulness of our approach by learning person- specific avatars from examples and then show how they can be animated in new poses and motions. We also show that the method can learn directly from raw scans with missing data, greatly simplifying the process of creating realistic avatars. Code is available for research purposes.
Project page Code arXiv Paper Supp Poster DOI URL BibTeX
Movement Generation and Control Conference Paper Nonlinear Stochastic Trajectory Optimization for Centroidal Momentum Motion Generation of Legged Robots Gazar, A., Khadiv, M., Kleff, S., DelPrete, A., Righetti, L. In Robotics Research, 420-435, Springer Proceedings in Advanced Robotics, 27, (Editors: Billard, Aude and Asfour, Tamim and Khatib, Oussama), Springer, Cham, 20th International Symposium on Robotics Research (ISRR 2022), September 2022 (Published)
Generation of robust trajectories for legged robots remains a challenging task due to the underlying nonlinear, hybrid and intrinsically unstable dynamics which needs to be stabilized through limited contact forces. Furthermore, disturbances arising from unmodelled contact interactions with the environment and model mismatches can hinder the quality of the planned trajectories leading to unsafe motions. In this work, we propose to use stochastic trajectory optimization for generating robust centroidal momentum trajectories to account for additive uncertainties on the model dynamics and parametric uncertainties on contact locations. Through an alternation between the robust centroidal and whole-body trajectory optimizations, we generate robust momentum trajectories while being consistent with the whole-body dynamics. We perform an extensive set of simulations subject to different uncertainties on a quadruped robot showing that our stochastic trajectory optimization problem reduces the amount of foot slippage for different gaits while achieving better performance over deterministic planning.
DOI URL BibTeX
Perceiving Systems Conference Paper Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors Wang, X., Li, G., Kuo, Y., Kocabas, M., Aksan, E., Hilliges, O. In 2022 International Conference on 3D Vision (3DV 2022), 353-362, IEEE, Piscataway, NJ, International Conference on 3D Vision (3DV 2022), September 2022 (Published)
We present a method for inferring diverse 3D models of human-object interactions from images. Reasoning about how humans interact with objects in complex scenes from a single 2D image is a challenging task given ambiguities arising from the loss of information through projection. In addition, modeling 3D interactions requires the generalization ability towards diverse object categories and interaction types. We propose an action-conditioned modeling of interactions that allows us to infer diverse 3D arrangements of humans and objects without supervision on contact regions or 3D scene geometry. Our method extracts high-level commonsense knowledge from large language models (such as GPT-3), and applies them to perform 3D reasoning of human-object interactions. Our key insight is priors extracted from large language models can help in reasoning about human-object contacts from textural prompts only. We quantitatively evaluate the inferred 3D models on a large human-object interaction dataset and show how our method leads to better 3D reconstructions. We further qualitatively evaluate the effectiveness of our method on real images and demonstrate its generalizability towards interaction types and object categories.
Project Page Video Arxiv DOI BibTeX
Physical Intelligence Article Scale-reconfigurable miniature ferrofluidic robots for negotiating sharply variable spaces Fan, X., Jiang, Y., Li, M., Zhang, Y., Tian, C., Mao, L., Xie, H., Sun, L., Yang, Z., Sitti, M. Science Advances, 8(37):eabq1677, September 2022 (Published) DOI BibTeX
Haptic Intelligence Conference Paper Towards Semi-Automated Pleural Cavity Access for Pneumothorax in Austere Environments L’Orsa, R., Lama, S., Westwick, D., Sutherland, G., Kuchenbecker, K. J. In Proceedings of the International Astronautical Congress (IAC), 1-7, Paris, France, September 2022 (Published)
Pneumothorax, a condition where injury or disease introduces air between the chest wall and lungs, can impede lung function and lead to respiratory failure and/or obstructive shock. Chest trauma from dynamic loads, hypobaric exposure from extravehicular activity, and pulmonary inflammation from celestial dust exposures could potentially cause pneumothoraces during spaceflight with or without exacerbation from deconditioning. On Earth, emergent cases are treated with chest tube insertion (tube thoracostomy, TT) when available, or needle decompression (ND) when not (i.e., pre-hospital). However, ND has high failure rates (up to 94\%), and TT has high complication rates (up to 37.9\%), especially when performed by inexperienced or intermittent operators. Thus neither procedure is ideal for a pure just-in-time training or skill refreshment approach, and both may require adjuncts for safe inclusion in Level of Care IV (e.g., short duration lunar orbit) or V (e.g., Mars transit) missions. Insertional complications are of particular concern since they cause inadvertent tissue damage that, while surgically repairable in an operating room, could result in (preventable) fatality in a spacecraft or other isolated, confined, or extreme (ICE) environments. Tools must be positioned and oriented correctly to avoid accidental insertion into critical structures, and they must be inserted no further than the thin membrane lining the inside of the rib cage (i.e., the parietal pleura). Operators identify pleural puncture via loss-of-resistance sensations on the tool during advancement, but experienced surgeons anecdotally describe a wide range of membrane characteristics: robust tissues require significant force to perforate, while fragile tissues deliver little-to-no haptic sensation when pierced. Both extremes can lead to tool overshoot and may be representative of astronaut tissues at the beginning (healthy) and end (deconditioned) of long duration exploration class missions. Given uncertainty surrounding physician astronaut selection criteria, skill retention, and tissue condition, an adjunct for improved insertion accuracy would be of value. We describe experiments conducted with an intelligent prototype sensorized system aimed at semi-automating tool insertion into the pleural cavity. The assembly would integrate with an in-mission medical system and could be tailored to fully complement an autonomous medical response agent. When coupled with minimal just-in-time training, it has the potential to bestow expert pleural access skills on non-expert operators without the use of ground resources, in both emergent and elective treatment scenarios.
URL BibTeX
Proceedings Towards semi-automated pleural cavity access for pneumothorax in austere environments L’Orsa, R., Lama, S., Westwick, D., Sutherland, G., Kuchenbecker, K. J. International Astronautical Congress (IAC), September 2022 BibTeX
Perceiving Systems Conference Paper InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction Huang, Y., Taheri, O., Black, M. J., Tzionas, D. In Pattern Recognition, 281-299, Lecture Notes in Computer Science, 13485, (Editors: Andres, Björn and Bernard, Florian and Cremers, Daniel and Frintrop, Simone and Goldlücke, Bastian and Ihrke, Ivo), Springer, Cham, 44th DAGM German Conference on Pattern Recognition (DAGM GCPR 2022), September 2022 (Published)
Humans constantly interact with daily objects to accomplish tasks. To understand such interactions, computers need to reconstruct these from cameras observing whole-body interaction with scenes. This is challenging due to occlusion between the body and objects, motion blur, depth/scale ambiguities, and the low image resolution of hands and graspable object parts. To make the problem tractable, the community focuses either on interacting hands, ignoring the body, or on interacting bodies, ignoring hands. The GRAB dataset addresses dexterous whole-body interaction but uses marker-based MoCap and lacks images, while BEHAVE captures video of body object interaction but lacks hand detail. We address the limitations of prior work with InterCap, a novel method that reconstructs interacting whole-bodies and objects from multi-view RGB-D data, using the parametric whole-body model SMPL-X and known object meshes. To tackle the above challenges, InterCap uses two key observations: (i) Contact between the hand and object can be used to improve the pose estimation of both. (ii) Azure Kinect sensors allow us to set up a simple multi-view RGB-D capture system that minimizes the effect of occlusion while providing reasonable inter-camera synchronization. With this method we capture the InterCap dataset, which contains 10 subjects (5 males and 5 females) interacting with 10 objects of various sizes and affordances, including contact with the hands or feet. In total, InterCap has 223 RGB-D videos, resulting in 67,357 multi-view frames, each containing 6 RGB-D images. Our method provides pseudo ground-truth body meshes and objects for each video frame. Our InterCap method and dataset fill an important gap in the literature and support many research directions. Our data and code are areavailable for research purposes.
Code Data YouTube Video pdf DOI URL BibTeX
Perceiving Systems Autonomous Learning Conference Paper InvGAN: Invertible GANs Ghosh, P., Zietlow, D., Black, M. J., Davis, L. S., Hu, X. In Pattern Recognition, 3-19, Lecture Notes in Computer Science, 13485, (Editors: Andres, Björn and Bernard, Florian and Cremers, Daniel and Frintrop, Simone and Goldlücke, Bastian and Ihrke, Ivo), Springer, Cham, 44th DAGM German Conference on Pattern Recognition (DAGM GCPR 2022), September 2022 (Published)
Generation of photo-realistic images, semantic editing and representation learning are only a few of many applications of high-resolution generative models. Recent progress in GANs have established them as an excellent choice for such tasks. However, since they do not provide an inference model, downstream tasks such as classification cannot be easily applied on real images using the GAN latent space. Despite numerous efforts to train an inference model or design an iterative method to invert a pre-trained generator, previous methods are dataset (e.g. human face images) and architecture (e.g. StyleGAN) specific. These methods are nontrivial to extend to novel datasets or architectures. We propose a general framework that is agnostic to architecture and datasets. Our key insight is that, by training the inference and the generative model together, we allow them to adapt to each other and to converge to a better quality model. Our InvGAN, short for Invertible GAN, successfully embeds real images in the latent space of a high quality generative model. This allows us to perform image inpainting, merging, interpolation and online data augmentation. We demonstrate this with extensive qualitative and quantitative experiments.
pdf DOI BibTeX
Perceiving Systems Conference Paper TEACH: Temporal Action Composition for 3D Humans Athanasiou, N., Petrovich, M., Black, M. J., Varol, G. In 2022 International Conference on 3D Vision (3DV), 414-423, 3DV'22, September 2022 (Published)
Given a series of natural language descriptions, our task is to generate 3D human motions that correspond semantically to the text, and follow the temporal order of the instructions. In particular, our goal is to enable the synthesis of a series of actions, which we refer to as temporal action composition. The current state of the art in text-conditioned motion synthesis only takes a single action or a single sentence as input. This is partially due to lack of suitable training data containing action sequences, but also due to the computational complexity of their non-autoregressive model formulation, which does not scale well to long sequences. In this work, we address both issues. First, we exploit the recent BABEL motion-text collection, which has a wide range of labeled actions, many of which occur in a sequence with transitions between them. Next, we design a Transformer-based approach that operates non-autoregressively within an action, but autoregressively within the sequence of actions. This hierarchical formulation proves effective in our experiments when compared with multiple baselines. Our approach, called TEACH for “TEmporal Action Compositions for Human motions”, produces realistic human motions for a wide variety of actions and temporal compositions from language descriptions. To encourage work on this new task, we make our code available for research purposes at teach.is.tue.mpg.de.
code arXiv website video camera-ready DOI URL BibTeX
Perceiving Systems Conference Paper TempCLR: Reconstructing Hands via Time-Coherent Contrastive Learning Ziani, A., Fan, Z., Kocabas, M., Christen, S., Hilliges, O. In 2022 International Conference on 3D Vision (3DV 2022), 627-636, IEEE, Piscataway, NJ, International Conference on 3D Vision (3DV 2022), September 2022 (Published)
We introduce TempCLR, a new time-coherent contrastive learning approach for the structured regression task of 3D hand reconstruction. Unlike previous time-contrastive methods for hand pose estimation, our framework considers temporal consistency in its augmentation scheme, and accounts for the differences of hand poses along the temporal direction. Our data-driven method leverages unlabelled videos and a standard CNN, without relying on synthetic data, pseudo-labels, or specialized architectures. Our approach improves the performance of fully-supervised hand reconstruction methods by 15.9% and 7.6% in PA-V2V on the HO-3D and FreiHAND datasets respectively, thus establishing new state-of-the-art performance. Finally, we demonstrate that our approach produces smoother hand reconstructions through time, and is more robust to heavy occlusions compared to the previous state-of-the-art which we show quantitatively and qualitatively.
Project Page Code Arxiv Video DOI URL BibTeX
Robotic Materials Patent Hydraulically Amplified Self-healing Electrostatic Actuators Keplinger, C. M., Acome, E. L., Kellaris, N. A., Mitchell, S. K. (US Patent 11408452), August 2022
An electro-hydraulic actuator includes a deformable shell defining an enclosed internal cavity and containing a liquid dielectric, first and second electrodes on first and second sides, respectively, of the enclosed internal cavity. An electrostatic force between the first and second electrodes upon application of a voltage to one of the electrodes draws the electrodes towards each other to displace the liquid dielectric within the enclosed internal cavity. The shell includes active and inactive areas such that the electrostatic forces between the first and second electrodes displaces the liquid dielectric within the enclosed internal cavity from the active area of the shell to the inactive area of the shell. The first and second electrodes, the deformable shell, and the liquid dielectric cooperate to form a self-healing capacitor, and the liquid dielectric is configured for automatically filling breaches in the liquid dielectric resulting from dielectric breakdown.
URL BibTeX
Robotic Materials Article A Pocket‐Sized Ten‐Channel High Voltage Power Supply for Soft Electrostatic Actuators Mitchell, S. K., Martin, T., Keplinger, C. Advanced Materials Technologies, 7(8), August 2022 (Published)
As soft electrostatic actuators find applications in bio-inspired robotics, compact and lightweight high voltage electronics that independently address many actuators are required. Here, a pocket-sized, battery-powered, 10-channel high voltage power supply (HVPS) is presented, which independently addresses each channel up to 10 kV. The HVPS uses one HV amplifier to create a HV rail and each output connects to the rail via custom optocouplers that are pulse-width modulated to vary their conductance. These optocouplers distribute charges to and from electrostatic devices at each output, creating a charge-controlled driving scheme that can generate independent and nearly arbitrary actuation waveforms for each channel. The HVPS weighs 250 g and measures 8.4 cm × 13.3 cm × 2 cm, about the size of a smartphone. The HVPS is characterized when driving hydraulically amplified self-healing electrostatic (HASEL) actuators. While powering a 5 nF actuator, the output of the HVPS reaches 8 kV in 100 ms and drives a 1.5 nF actuator at 100 Hz (0 to 5.4 kV). The HVPS powers an active surface consisting of an array of HASELs and generates undulatory locomotion of a soft robotic inchworm, highlighting the potential for compact HV electronics that power multi-degree-of-freedom robotic systems based on electrostatic devices.
DOI URL BibTeX
Physical Intelligence Article Adaptive wireless millirobotic locomotion into distal vasculature Wang, T., Ugurlu, H., Yan, Y., Li, M., Li, M., Wild, A., Yildiz, E., Schneider, M., Sheehan, D., Hu, W., Sitti, M. Nature Communications, 13(1):1-17, August 2022 DOI BibTeX
Empirical Inference Conference Paper Can Large Language Models Distinguish Cause from Effect? Lyu, Z., Jin, Z., Mihalcea, R., Sachan, M., Schölkopf, B. UAI 2022 Workshop on Causal Representation Learning, August 2022 (Published) URL BibTeX
Physics for Inference and Optimization Article Community detection and reciprocity in networks by jointly modeling pairs of edges Contisciani, M., Safdari, H., De Bacco, C. Journal of Complex Networks, 10(4):cnac034, August 2022 (Published) Code Preprint DOI URL BibTeX
Empirical Inference Conference Paper Learning soft interventions in complex equilibrium systems Besserve, M., Schölkopf, B. Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, 180:170-180, Proceedings of Machine Learning Research, (Editors: Cussens, James and Zhang, Kun), PMLR, The 38th Conference on Uncertainty in Artificial Intelligence (UAI 2022) , August 2022 (Published) URL BibTeX
Physical Intelligence Article Magnetically actuated gearbox for the wireless control of millimeter-scale robots Hong, C., Ren, Z., Wang, C., Li, M., Wu, Y., Tang, D., Hu, W., Sitti, M. Science Robotics, 7(69), August 2022 (Published) DOI BibTeX
Movement Generation and Control Empirical Inference Article Model-free reinforcement learning for robust locomotion using demonstrations from trajectory optimization Bogdanovic, M., Khadiv, M., Righetti, L. Frontiers in Robotics and AI, 9, August 2022 (Published) DOI BibTeX
Materials Article Multifunctional Self-Cross-Linked Copolymer Binder for High-Loading Silicon Anodes Niesen, S., Fox, A., Murugan, S., Richter, G., Buchmeiser, M. R. ACS Applied Energy Materials, 5(9):11386-11391, August 2022 (Published) DOI BibTeX
Empirical Inference Master Thesis On the Adversarial Robustness of Causal Algorithmic Recourse Dominguez-Olmedo, R. University of Tübingen, Germany, August 2022 (Published) BibTeX
Physical Intelligence Article On-demand anchoring of wireless soft miniature robots on soft surfaces Soon, R. H., Ren, Z., Hu, W., Bozuyuk, U., Yildiz, E., Li, M., Sitti, M. Proceedings of the National Academy of Sciences, 119(34):e2207767119, August 2022 (Published) DOI BibTeX
Empirical Inference Article Optimal Client Sampling for Federated Learning Chen, W., Horváth, S., Richtárik, P. Transactions on Machine Learning Research, August 2022 (Published) URL BibTeX
Social Foundations of Computation Book Patterns, Predictions, and Actions: Foundations of Machine Learning Hardt, M., Recht, B. Princeton University Press, August 2022 (Published)
An authoritative, up-to-date graduate textbook on machine learning that highlights its historical context and societal impacts Patterns, Predictions, and Actions introduces graduate students to the essentials of machine learning while offering invaluable perspective on its history and social implications. Beginning with the foundations of decision making, Moritz Hardt and Benjamin Recht explain how representation, optimization, and generalization are the constituents of supervised learning. They go on to provide self-contained discussions of causality, the practice of causal inference, sequential decision making, and reinforcement learning, equipping readers with the concepts and tools they need to assess the consequences that may arise from acting on statistical decisions. Provides a modern introduction to machine learning, showing how data patterns support predictions and consequential actions Pays special attention to societal impacts and fairness in decision making Traces the development of machine learning from its origins to today Features a novel chapter on machine learning benchmarks and datasets Invites readers from all backgrounds, requiring some experience with probability, calculus, and linear algebra An essential textbook for students and a guide for researchers
URL BibTeX
Physical Intelligence Article Piezo Capsule: Ultrasonic Way of Wireless Pressure Measurement Mutlu, S., Aghakhani, A., Sitti, M. Advanced Intelligent Systems, 4(10):2200125, August 2022 (Published) DOI BibTeX
Dynamic Locomotion Conference Paper Power to the springs: Passive elements are sufficient to drive push-off in human walking Buchmann, A., Kiss, B., Badri-Spröwitz, A., Renjewski, D. In Robotics in Natural Settings , 21-32, Lecture Notes in Networks and Systems, 530, (Editors: Cascalho, José M. and Tokhi, Mohammad Osman and Silva, Manuel F. and Mendes, Armando and Goher, Khaled and Funk, Matthias), Springer, Cham, 25th International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machine (CLAWAR 2022), August 2022 (Published)
For the impulsive ankle push-off (APO) observed in human walking two muscle-tendon-units (MTUs) spanning the ankle joint play an important role: Gastrocnemius (GAS) and Soleus (SOL). GAS and SOL load the Achilles tendon to store elastic energy during stance followed by a rapid energy release during APO. We use a neuromuscular simulation (NMS) and a bipedal robot to investigate the role of GAS and SOL on the APO. We optimize the simulation for a robust gait and then sequentially replace the MTUs of (1) GAS, (2) SOL and (3) GAS and SOL by linear springs. To validate the simulation, we implement NMS-3 on a bipedal robot. Simulation and robot walk steady for all trials showing an impulsive APO. Our results imply that the elastic MTU properties shape the impulsive APO. For prosthesis or robot design that is, no complex ankle actuation is needed to obtain an impulsive APO, if more mechanical intelligence is incorporated in the design.
DOI URL BibTeX
Haptic Intelligence Miscellaneous Predicting Knee Adduction Moment Response to Gait Retraining Rokhmanova, N., Kuchenbecker, K. J., Shull, P. B., Ferber, R., Halilaj, E. Extended abstract presented at North American Congress of Biomechanics (NACOB), Ottawa, Canada, August 2022 (Published)
Personalized gait retraining has shown promise as a conservative intervention for slowing knee osteoarthritis (OA) progression [1,2]. Changing the foot progression angle is an easy-to-learn gait modification that often reduces the knee adduction moment (KAM), a correlate of medial joint loading. Deployment to clinics is challenging, however, because customizing gait retraining still requires gait lab instrumentation. Innovation in wearable sensing and vision-based motion tracking could bring lab-level accuracy to the clinic, but current markerless motion-tracking algorithms cannot accurately assess if gait retraining will reduce someone's KAM by a clinically meaningful margin. To assist clinicians in determining if a patient will benefit from toe-in gait, we built a predictive model to estimate KAM reduction using only measurements that can be easily obtained in the clinic.
BibTeX
Empirical Inference Article Probabilistic solvers enable a straight-forward exploration of numerical uncertainty in neuroscience models Oesterle, J., Krämer, N., P., H., Berens, P. Journal of Computational Neuroscience, 50(4):485-503, August 2022 (Published) DOI BibTeX
Empirical Inference Conference Paper Probing the Robustness of Independent Mechanism Analysis for Representation Learning Sliwa, J., Ghosh, S., Stimper, V., Gresele, L., Schölkopf, B. UAI 2022 Workshop on Causal Representation Learning (UAI CRL 2022), 1st Workshop on Causal Representation Learning at the 38th Conference on Uncertainty in Artificial Intelligence (UAI CRL 2022) , August 2022 (Published) arXiv URL BibTeX
Empirical Inference Article Real Time Landmark Detection for Within- and Cross Subject Tracking With Minimal Human Supervision Frueh, M., Schilling, A., Gatidis, S., Kuestner, T. IEEE Access, 10:81192-81202, August 2022 (Published) DOI BibTeX
Empirical Inference Article Semi-Supervised and Unsupervised Deep Visual Learning: A Survey Chen, Y., Mancini, M., Zhu, X., Akata, Z. IEEE Transactions on Pattern Analysis and Machine Intelligence, August 2022, *early access (Published) DOI BibTeX
Physical Intelligence Bioinspired Autonomous Miniature Robots Article Soft-robotic ciliated epidermis for reconfigurable coordinated fluid manipulation Ren, Z., Zhang, M., Song, S., Liu, Z., Hong, C., Wang, T., Dong, X., Hu, W., Sitti, M. Science Advances, 8(34):eabq2345, August 2022 (Published) DOI BibTeX
Physical Intelligence Article Wireless Miniature Magnetic Phase-Change Soft Actuators Tang, Y., Li, M., Wang, T., Dong, X., Hu, W., Sitti, M. Advanced Materials, 34(40):2204185, August 2022 (Published) DOI BibTeX
Haptic Intelligence Conference Paper Wrist-Squeezing Force Feedback Improves Accuracy and Speed in Robotic Surgery Training Machaca, S., Cao, E., Chi, A., Adrales, G., Kuchenbecker, K. J., Brown, J. D. In Proceedings of the IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), Seoul, South Korea, August 2022 (Published)
Current robotic minimally invasive surgery (RMIS) platforms provide surgeons with no haptic feedback of the robot's physical interactions. This limitation forces surgeons to rely heavily on visual feedback and can make it challenging for surgical trainees to manipulate tissue gently. Prior research has demonstrated that haptic feedback can increase task accuracy in RMIS training. However, it remains unclear whether these improvements represent a fundamental improvement in skill, or if they simply stem from re-prioritizing accuracy over task completion time. In this study, we provide haptic feedback of the force applied by the surgical instruments using custom wrist-squeezing devices. We hypothesize that individuals receiving haptic feedback will increase accuracy (produce less force) while increasing their task completion time, compared to a control group receiving no haptic feedback. To test this hypothesis, N=21 novice participants were asked to repeatedly complete a ring rollercoaster surgical training task as quickly as possible. Results show that participants receiving haptic feedback apply significantly less force (0.67 N) than the control group, and they complete the task no faster or slower than the control group after twelve repetitions. Furthermore, participants in the feedback group decreased their task completion times significantly faster (7.68 %) than participants in the control group (5.26 %). This form of haptic feedback, therefore, has the potential to help trainees improve their technical accuracy without compromising speed.
DOI BibTeX
Perceiving Systems Article iRotate: Active visual SLAM for omnidirectional robots Bonetto, E., Goldschmid, P., Pabst, M., Black, M. J., Ahmad, A. Robotics and Autonomous Systems, 154:104102, Elsevier, August 2022 (Published)
In this paper, we present an active visual SLAM approach for omnidirectional robots. The goal is to generate control commands that allow such a robot to simultaneously localize itself and map an unknown environment while maximizing the amount of information gained and consuming as low energy as possible. Leveraging the robot’s independent translation and rotation control, we introduce a multi-layered approach for active V-SLAM. The top layer decides on informative goal locations and generates highly informative paths to them. The second and third layers actively re-plan and execute the path, exploiting the continuously updated map and local features information. Moreover, we introduce two utility formulations to account for the presence of obstacles in the field of view and the robot’s location. Through rigorous simulations, real robot experiments, and comparisons with state-of-the-art methods, we demonstrate that our approach achieves similar coverage results with lesser overall map entropy. This is obtained while keeping the traversed distance up to 39% shorter than the other methods and without increasing the wheels’ total rotation amount. Code and implementation details are provided as open-source and all the generated data is available online for consultation.
Code Data iRotate Data Independent Camera experiments DOI URL BibTeX
Empirical Inference Conference Paper A Hierarchical Model of Attention over Time Kim, J., Singh, S., Yurovsky, D., Fisher, A. A. E. In Proceedings of the 44th Annual Meeting of the Cognitive Science Society (Cogsci 2022), 350-357 , (Editors: J. Culbertson and A. Perfors and H. Rabagliati and V. Ramenzoni), 44st Annual Meeting of the Cognitive Science Society (CogSci 2022) , July 2022 (Published) URL BibTeX
Empirical Inference Conference Paper Action-Sufficient State Representation Learning for Control with Structural Constraints Huang*, B., Lu*, C., Leqi, L., Hernandez-Lobato, J. M., Glymour, C., Schölkopf, B., Zhang, K. Proceedings of the 39th International Conference on Machine Learning (ICML), 162:9260-9279, (Editors: Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan), PMLR, July 2022, *equal contribution (Published) URL BibTeX
Empirical Inference Conference Paper Adapting the Linearised Laplace Model Evidence for Modern Deep Learning Antoran, J., Janz, D., Allingham, J. U., Daxberger, E., Barbano, R. R., Nalisnick, E., Hernandez-Lobato, J. M. Proceedings of the 39th International Conference on Machine Learning (ICML), 162:796-821, (Editors: Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan), PMLR, July 2022 (Published) URL BibTeX
Rationality Enhancement Article Boosting human decision-making with AI-generated decision aids Becker, F., Skirzyński, J., van Opheusden, B., Lieder, F. Computational Brain & Behavior, 5(4):467-490, July 2022 (Published)
Human decision-making is plagued by many systematic errors. Many of these errors can be avoided by providing decision aids that guide decision-makers to attend to the important information and integrate it according to a rational decision strategy. Designing such decision aids is a tedious manual process. Advances in cognitive science might make it possible to automate this process in the future. We recently introduced machine learning methods for discovering optimal strategies for human decision-making automatically and an automatic method for explaining those strategies to people. Decision aids constructed by this method were able to improve human decision-making. However, following the descriptions generated by this method is very tedious. We hypothesized that this problem can be overcome by conveying the automatically discovered decision strategy as a series of natural language instructions for how to reach a decision. Experiment 1 showed that people do indeed understand such procedural instructions more easily than the decision aids generated by our previous method. Encouraged by this finding, we developed an algorithm for translating the output of our previous method into procedural instructions. We applied the improved method to automatically generate decision aids for a naturalistic planning task (i.e., planning a road trip) and a naturalistic decision task (i.e., choosing a mortgage). Experiment 2 showed that these automatically generated decision-aids significantly improved people's performance in planning a road trip and choosing a mortgage. These findings suggest that AI-powered boosting has potential for improving human decision-making in the real world.
DOI BibTeX