Institute Homepage
Institute Homepage DE Sign In

Publications

DEPARTMENTS

Emperical Interference

Haptic Intelligence

Modern Magnetic Systems

Perceiving Systems

Physical Intelligence

Robotic Materials

Social Foundations of Computation

Research Groups

Autonomous Vision

Autonomous Learning

Bioinspired Autonomous Miniature Robots

Dynamic Locomotion

Embodied Vision

Human Aspects of Machine Learning

Intelligent Control Systems

Learning and Dynamical Systems

Locomotion in Biorobotic and Somatic Systems

Micro, Nano, and Molecular Systems

Movement Generation and Control

Neural Capture and Synthesis

Physics for Inference and Optimization

Organizational Leadership and Diversity

Probabilistic Learning Group

Topics

Robot Learning

Conference Paper

2022

Autonomous Learning

Robotics

AI

Career

Award

Intelligent Control Systems Autonomous Motion Article Safe and Fast Tracking on a Robot Manipulator: Robust MPC and Neural Network Control Nubert, J., Koehler, J., Berenz, V., Allgower, F., Trimpe, S. IEEE Robotics and Automation Letters, 5(2):3050-3057, 2020 (Published)
Fast feedback control and safety guarantees are essential in modern robotics. We present an approach that achieves both by combining novel robust model predictive control (MPC) with function approximation via (deep) neural networks (NNs). The result is a new approach for complex tasks with nonlinear, uncertain, and constrained dynamics as are common in robotics. Specifically, we leverage recent results in MPC research to propose a new robust setpoint tracking MPC algorithm, which achieves reliable and safe tracking of a dynamic setpoint while guaranteeing stability and constraint satisfaction. The presented robust MPC scheme constitutes a one-layer approach that unifies the often separated planning and control layers, by directly computing the control command based on a reference and possibly obstacle positions. As a separate contribution, we show how the computation time of the MPC can be drastically reduced by approximating the MPC law with a NN controller. The NN is trained and validated from offline samples of the MPC, yielding statistical guarantees, and used in lieu thereof at run time. Our experiments on a state-of-the-art robot manipulator are the first to show that both the proposed robust and approximate MPC schemes scale to real-world robotic systems.
arXiv PDF DOI BibTeX
Empirical Inference Conference Paper Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation Lu, C., Huang, B., Wang, K., Hernández-Lobato, J. M., Zhang, K., Schölkopf, B. Offline Reinforcement Learning - Workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS), 2020 (Published) arXiv URL BibTeX
Autonomous Learning Embodied Vision Conference Paper Sample-efficient Cross-Entropy Method for Real-time Planning Pinneri, C., Sawant, S., Blaes, S., Achterhold, J., Stueckler, J., Rolinek, M., Martius, G. In Conference on Robot Learning 2020, 2020 (Published)
Trajectory optimizers for model-based reinforcement learning, such as the Cross-Entropy Method (CEM), can yield compelling results even in high-dimensional control tasks and sparse-reward environments. However, their sampling inefficiency prevents them from being used for real-time planning and control. We propose an improved version of the CEM algorithm for fast planning, with novel additions including temporally-correlated actions and memory, requiring 2.7-22x less samples and yielding a performance increase of 1.2-10x in high-dimensional control problems.
Paper Code Spotlight-Video URL BibTeX
Physical Intelligence Article Selection for function: from chemically synthesized prototypes to 3D-printed microdevices Bachmann, F., Giltinan, J., Codutti, A., Klumpp, S., Sitti, M., Faivre, D. Advanced Intelligent Systems, 2(10):2000078, 2020 DOI BibTeX
Physical Intelligence Article Selectively controlled magnetic microrobots with opposing helices Giltinan, J., Katsamba, P., Wang, W., Lauga, E., Sitti, M. Applied Physics Letters, 116(13):134101, 2020 (Published) DOI BibTeX
Theory of Inhomogeneous Condensed Matter Article Self-motility of an active particle induced by correlations in the surrounding solution Domínguez, A., Popescu, M. N., Rohwer, C. M., Dietrich, S. Physical Review Letters, 125(26):268002, 2020 DOI URL BibTeX
Autonomous Vision Article Self-supervised motion deblurring Liu, P., Janai, J., Pollefeys, M., Sattler, T., Geiger, A. IEEE Robotics and Automation Letters, 2020
Motion blurry images challenge many computer vision algorithms, e.g., feature detection, motion estimation, or object recognition. Deep convolutional neural networks are state-of-the-art for image deblurring. However, obtaining training data with corresponding sharp and blurry image pairs can be difficult. In this paper, we present a differentiable reblur model for self-supervised motion deblurring, which enables the network to learn from real-world blurry image sequences without relying on sharp images for supervision. Our key insight is that motion cues obtained from consecutive images yield sufficient information to inform the deblurring task. We therefore formulate deblurring as an inverse rendering problem, taking into account the physical image formation process: we first predict two deblurred images from which we estimate the corresponding optical flow. Using these predictions, we re-render the blurred images and minimize the difference with respect to the original blurry inputs. We use both synthetic and real dataset for experimental evaluations. Our experiments demonstrate that self-supervised single image deblurring is really feasible and leads to visually compelling results.
pdf Project Page Blog BibTeX
Article Shape-Conformable, Eco-Friendly Cellulose Aerogels as High-Performance Battery Separators Raafat, L., Wicklein, B., Majer, G., Jahnke, T., Diem, A. M., Bill, J., Burghard, Z. ACS Applied Energy Materials, 4(1):763-774, American Chemical Society, Washington, DC, 2020 DOI BibTeX
Modern Magnetic Systems Article Single shot acquisition of spatially resolved spin wave dispersion relations using X-ray microscopy Träger, N., Groß, F., Förster, J., Baumgaertl, K., Stoll, H., Weigand, M., Schütz, G., Grundler, D., Gräfe, J. Scientific Reports, 10:18146, Nature Publishing Group, London, UK, 2020 DOI BibTeX
Modern Magnetic Systems Article Single-frame far-field diffractive imaging with randomized illumination Levitan, A. L., Keskinbora, K., Sanli, U. T., Weigand, M., Comin, R. Optics Express, 28(25):37103-37117, Optical Society of America, Washington, DC, 2020 DOI BibTeX
Modern Magnetic Systems Article Skyrmion lattice phases in thin film multilayer Zázvorka, J., Dittrich, F., Ge, Y., Kerber, N., Raab, K., Winkler, T., Litzius, K., Veis, M., Virnau, P., Kläui, M. Advanced Functional Materials, 30(46), Wiley-VCH Verlag GmbH, Weinheim, 2020 DOI BibTeX
Theory of Inhomogeneous Condensed Matter Article Smoluchowski equations for linker-mediated irreversible aggregation Tavares, J. M., Antunes, G. C., Dias, C. S., Telo da Gama, M. M., Araujo, N. A. M. Soft Matter, 16(32):7513-7523, Royal Society of Chemistry, Cambridge, UK, 2020 DOI BibTeX
Micro, Nano, and Molecular Systems Book Chapter Soft Microrobots Based on Photoresponsive Materials Palagi, S. In Mechanically Responsive Materials for Soft Robotics, 327-362, (Editors: Koshima, Hideko), Wiley-VCH, Weinheim, 2020 DOI BibTeX
Intelligent Control Systems Article Spatial Scheduling of Informative Meetings for Multi-Agent Persistent Coverage Haksar, R. N., Trimpe, S., Schwager, M. IEEE Robotics and Automation Letters, 5(2):3027-3034, 2020 (Published) DOI BibTeX
Modern Magnetic Systems Article Specific isotope-responsive breathing transition in flexible metal-organic frameworks Kim, J. Y., Park, J., Ha, J., Jung, M., Wallacher, D., Franz, A., Balderas-Xicohténcatl, R., Hirscher, M., Kang, S. G., Park, J. T., Oh, I. H., Moon, H. R., Oh, H. Journal of the American Chemical Society, 142(31):13278-13282, American Chemical Society, Washington, DC, 2020 DOI BibTeX
Physical Intelligence Article Sperm Cell Driven Microrobots-Emerging Opportunities and Challenges for Biologically Inspired Robotic Design Singh, A. V., Ansari, M. H. D., Mahajan, M., Srivastava, S., Kashyap, S., Dwivedi, P., Pandit, V., Katha, U. Micromachines, 11(4):448, 2020 DOI BibTeX
Theory of Inhomogeneous Condensed Matter Article Storage of Energy in Constrained Non-Equilibrium Systems Zhang, Y., Gizynski, K., Maciolek, A., Holyst, R. Entropy, 22(5):557, MDPI, Basel, 2020 DOI BibTeX
Empirical Inference Article Stratification of behavioral response to transcranial current stimulation by resting-state electrophysiology Mastakouri, A. A. bioRxiv, Cold Spring Harbor Laboratory, 2020 (Published) DOI URL BibTeX
Empirical Inference Conference Paper Structured policy representation: Imposing stability in arbitrarily conditioned dynamic systems Urain, J., Tateo, D., Ren, T., Peters, J. 3rd Robot Learning Workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS), 2020 (Published) URL BibTeX
Micro, Nano, and Molecular Systems Article Submicron Particle Concentration and Patterning with Ultralow Frequency Acoustic Vibration Zhou, Y., Ma, Z., Ai, Y. Analytical Chemistry, 92(19):12795-12800, 2020 (Published) DOI BibTeX
Materials Article Suppressing instabilities in defect-scarce nanowires by controlling the energy release rate during incipient plasticity Shin, J. G., Richter, G., Gianola, D. S. Materials & Design, 189:108460, 2020 DOI BibTeX
Embodied Vision Book Chapter TUM Flyers: Vision-Based MAV Navigation for Systematic Inspection of Structures Usenko, V., Stumberg, L. V., Stückler, J., Cremers, D. In Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users: The Experience of the European Robotics Challenges, 136:189-209, Springer Tracts in Advanced Robotics, Springer International Publishing, 2020 (Published) DOI URL BibTeX
Materials Article Tantalum and zirconium induced structural transitions at complex [111] tilt grain boundaries in copper Meiners, T., Duarte, J. M., Richter, G., Dehm, G., Liebscher, C. H. Acta Materialia, 190:93-104, 2020 DOI BibTeX
Optics and Sensing Laboratory Article Temporal variations in rockfall and rock-wall retreat rates in a deglaciated valley over the past 11 k.y. Mohadjer, S., Ehlers, T. A., Nettesheim, M., Ott, M. B., Glotzbach, C., Drews, R. Geology, 48(6):594-598, 2020 (Published) DOI URL BibTeX
Probabilistic Learning Group Article Text Normalization Using Encoder-Decoder Networks Based on the Causal Feature Extractor Javaloy, A. G. G. Applied Sciences, 10(13):4551, 2020 (Published) DOI BibTeX
Physical Intelligence Article The effect of substrate wettability and modulus on gecko and gecko-inspired synthetic adhesion in variable temperature and humidity Mitchell, C. T., Dayan, C. B., Drotlef, D., Sitti, M., Stark, A. Y. Scientific Reports, 10:19748, 2020 DOI URL BibTeX
Modern Magnetic Systems Article The role of temperature and drive current in skyrmion dynamics Litzius, K., Leliaert, J., Bassirian, P., Rodrigues, D., Kromin, S., Lemesh, I., Zazvorka, J., Lee, K., Mulkers, J., Kerber, N., Heinze, D., Keil, N., Reeve, R. M., Weigand, M., Van Waeyenberge, B., Schütz, G., Everschor-Sitte, K., Beach, G. S. D., Kläui, M. Nature Electronics, 3(1):30-36, Springer Nature, London, 2020 DOI BibTeX
Physical Intelligence Article Thermal effects on the crystallization kinetics, and interfacial adhesion of single-crystal phase-change gallium Yunusa, M., Lahlou, A., Sitti, M. Advanced Materials, 32(10):1907453, 2020 (Published)
Although substrates play an important role upon crystallization of supercooled liquids, the influences of surface temperature and thermal property have remained elusive. Here, the crystallization of supercooled phase‐change gallium (Ga) on substrates with different thermal conductivity is studied. The effect of interfacial temperature on the crystallization kinetics, which dictates thermo‐mechanical stresses between the substrate and the crystallized Ga, is investigated. At an elevated surface temperature, close to the melting point of Ga, an extended single‐crystal growth of Ga on dielectric substrates due to layering effect and annealing is realized without the application of external fields. Adhesive strength at the interfaces depends on the thermal conductivity and initial surface temperature of the substrates. This insight can be applicable to other liquid metals for industrial applications, and sheds more light on phase‐change memory crystallization.
DOI BibTeX
Materials Article Thermal oxidation of amorphous CuxZr1-x alloys: Role of composition-dependent thermodynamic stability Xu, Y., Chen, Y., Schützendübe, P., Zhu, S., Huang, Y., Ma, Z., Liu, Y., Wang, Z. Applied Surface Science, 503:144376, Elsevier B.V., Amsterdam, 2020 DOI BibTeX
Modern Magnetic Systems Article Time-resolved visualization of the magnetization canting induced by field-like spin–orbit torques Finizio, S., Wintz, S., Mayr, S., Huxtable, A. J., Langer, M., Bailey, J., Burnell, G., Marrows, C. H., Raabe, J. Applied Physics Letters, 117(21), American Institute of Physics, Melville, NY, 2020 DOI BibTeX
Statistical Learning Theory Conference Paper Too Relaxed to Be Fair Lohaus, M., Perrot, M., von Luxburg, U. Proceedings of the 37th International Conference on Machine Learning (ICML 2020), 119:6316 - 6325, Proceedings of Machine Learning Research, Curran Associates, Inc. , Red Hook, International Conference of Machine Learning (ICML), 2020 (Published) URL BibTeX
Materials Article Toward Standardized Photocatalytic Oxygen Evolution Rates Using RuO2@TiO2 as a Benchmark Vignolo-González, H. A., Laha, S., Jiménez-Solano, A., Oshima, T., Duppel, V., Schützendübe, P., Lotsch, B. V. Matter, 3(2):464-486, Cell Press, Maryland Heights, MO, 2020 DOI BibTeX
Theory of Inhomogeneous Condensed Matter Article Toward a density-functional theory for the Jagla fluid Gußmann, F., Dietrich, S., Roth, R. Physical Review E, 102(6):062112, American Physical Society, Melville, NY, 2020 DOI BibTeX
Physical Intelligence Conference Paper Towards 5-DoF control of an untethered magnetic millirobot via MRI gradient coils Erin, O., Antonelli, D., Tiryaki, M. E., Sitti, M. In 2020 IEEE International Conference on Robotics and Automation (ICRA 2020), 6551-6557, IEEE, Piscataway, NJ, IEEE International Conference on Robotics and Automation (ICRA 2020), 2020 DOI BibTeX
Autonomous Vision Conference Paper Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis Liao, Y., Schwarz, K., Mescheder, L., Geiger, A. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 5870 - 5879, IEEE, Piscataway, NJ, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2020 (Published)
In recent years, Generative Adversarial Networks have achieved impressive results in photorealistic image synthesis. This progress nurtures hopes that one day the classical rendering pipeline can be replaced by efficient models that are learned directly from images. However, current image synthesis models operate in the 2D domain where disentangling 3D properties such as camera viewpoint or object pose is challenging. Furthermore, they lack an interpretable and controllable representation. Our key hypothesis is that the image generation process should be modeled in 3D space as the physical world surrounding us is intrinsically three-dimensional. We define the new task of 3D controllable image synthesis and propose an approach for solving it by reasoning both in 3D space and in the 2D image domain. We demonstrate that our model is able to disentangle latent 3D factors of simple multi-object scenes in an unsupervised fashion from raw images. Compared to pure 2D baselines, it allows for synthesizing scenes that are consistent wrt. changes in viewpoint or object pose. We further evaluate various 3D representations in terms of their usefulness for this challenging task.
pdf suppmat Video 2 Project Page Video 1 Slides Poster DOI BibTeX
Theory of Inhomogeneous Condensed Matter Article Tracer diffusion on a crowded random Manhattan lattice Mejía-Monasterio, C., Nechaev, S., Oshanin, G., Vasilyev, O. New Journal of Physics, 22(3):033024, IOP Publishing, Bristol, 2020 DOI BibTeX
Empirical Inference Article Training deep neural density estimators to identify mechanistic models of neural dynamics Gonçalves, P. J., Lueckmann, J., Deistler, M., Nonnenmacher, M., Öcal, K., Bassetto, G., Chintaluri, C., Podlaski, W. F., Haddad, S. A., Vogels, T. P., Greenberg, D. S., Macke, J. H. eLife, 9:article no. e56261, (Editors: Huguenard, John R. and O’Leary, Timothy and Goldman, Mark S.), 2020 (Published) DOI BibTeX
Physical Intelligence Robotics Article Twisting and untwisting of twisted nematic elastomers Davidson, Z. S., Kapernaum, N., Fiene, J., Giesselmann, F., Sitti, M. Physical Review Materials, 4(10):105601, 2020 DOI URL BibTeX
Statistical Learning Theory Article Two-sample Hypothesis Testing for Inhomogeneous Random Graphs Ghoshdastidar, D., Gutzeit, M., Carpentier, A., von Luxburg, U. Annals of Statistics, 48(4):2208-2229, 2020 BibTeX
Micro, Nano, and Molecular Systems Article Ultrasound-assisted cyanide extraction of gold from gold concentrate at low temperature Yu, S., Yu, T., Song, W., Yu, X., Qiao, J., Wang, W., Dong, H., Wu, Z., Dai, L., Li, T. Ultrasonics Sonochemistry, 64:105039, 2020 (Published) DOI BibTeX
Physical Intelligence Bioinspired Autonomous Miniature Robots Article Ultrasound-guided wireless tubular robotic anchoring system Wang, T., Hu, W., Ren, Z., Sitti, M. IEEE Robotics and Automation Letters, 5(3):4859-4866, IEEE, 2020 DOI URL BibTeX
Article Unusual Iron Nitride Formation Upon Nitriding Fe-Si Alloy Meka, S. R., Schubert, A., Bischoff, E., Mittemeijer, E. J. Metallurgical and Materials Transactions A, 51(6):3154-3166, Springer Sciences & Business Media, New York, NY, 2020 DOI BibTeX
Materials Article Very low-temperature growth of few-layer graphene by Ni-induced crystallization of amorphous carbon in vacuum Chen, Y., Wang, J., Schützendübe, P., Wang, Z., Mittemeijer, E. J. Carbon, 159:37-44, Elsevier, Amsterdam, 2020 DOI BibTeX
Embodied Vision Article Visual-Inertial Mapping with Non-Linear Factor Recovery Usenko, V., Demmel, N., Schubert, D., Stückler, J., Cremers, D. IEEE Robotics and Automation Letters (RA-L), 5(2):422-429, 2020, presented at IEEE International Conference on Robotics and Automation (ICRA) 2020, preprint arXiv:1904.06504 (Published)
Cameras and inertial measurement units are complementary sensors for ego-motion estimation and environment mapping. Their combination makes visual-inertial odometry (VIO) systems more accurate and robust. For globally consistent mapping, however, combining visual and inertial information is not straightforward. To estimate the motion and geometry with a set of images large baselines are required. Because of that, most systems operate on keyframes that have large time intervals between each other. Inertial data on the other hand quickly degrades with the duration of the intervals and after several seconds of integration, it typically contains only little useful information. In this paper, we propose to extract relevant information for visual-inertial mapping from visual-inertial odometry using non-linear factor recovery. We reconstruct a set of non-linear factors that make an optimal approximation of the information on the trajectory accumulated by VIO. To obtain a globally consistent map we combine these factors with loop-closing constraints using bundle adjustment. The VIO factors make the roll and pitch angles of the global map observable, and improve the robustness and the accuracy of the mapping. In experiments on a public benchmark, we demonstrate superior performance of our method over the state-of-the-art approaches.
Code Preprint URL BibTeX
Physical Intelligence Locomotion in Biorobotic and Somatic Systems Article Wearable and stretchable strain sensors: materials, sensing mechanisms, and applications Souri, H., Banerjee, H., Jusufi, A., Radacsi, N., Stokes, A. A., Park, I., Sitti, M., Amjadi, M. Advanced Intelligent Systems, 2(8):2000039, 2020 DOI URL BibTeX
Theory of Inhomogeneous Condensed Matter Article Wetting transitions on soft substrates Napiorkowski, M., Schimmele, L., Dietrich, S. EPL, 129(1):16002, EDP Science, Les-Ulis, 2020 DOI BibTeX
Physical Intelligence Article Zwitterionic 3D‐printed non‐immunogenic stealth microrobots Cabanach, P., Pena-Francesch, A., Sheehan, D., Bozuyuk, U., Yasa, O., Borros, S., Sitti, M. Advanced Materials, 32(42):2003013, 2020 DOI URL BibTeX
Empirical Inference Article sbi: A toolkit for simulation-based inference Tejero-Cantero, A., Boelts, J., Deistler, M., Lueckmann, J., Durkan, C., Gonçalves, P. J., Greenberg, D. S., Macke, J. H. Journal of Open Source Software, 5(52):article no. 2505, 2020 (Published) DOI BibTeX
Miscellaneous 19F MR-based Quantitative Method for Determination of Ca(II) Using Lanthanide Complexes Gambino, G., Gambino, T., Pohmann, R., Angelovski, G. 15th European Molecular Imaging Meeting (EMIM 2020), 2020 BibTeX
Miscellaneous 31P Transversal Relaxation Times in the Human Brain at 9.4T Dorst, J., Borbath, T., Ruhm, L., Avdievich, N., Henning, A. 2020 ISMRM & SMRT Virtual Conference & Exhibition, 2020
{31P transversal relaxation times in the human brain at 9.4T are reported. These values are useful to optimize measurement protocols, and to perform absolute quantification. Measurements were performed using a STEAM sequence. To account for J-evolution of homonuclear spin-spin coupled metabolites, basis sets were modeled in VeSPA and spectra were fitted in LCModel. The measured T2 relaxation times are between 93ms and 116ms for phosphomonoesters and \textendashdiesters and PCr, and between 25ms and 45ms for Piintra, ATP and tNAD.}
BibTeX