Institute Homepage
Institute Homepage DE Sign In

Publications

DEPARTMENTS

Emperical Interference

Haptic Intelligence

Modern Magnetic Systems

Perceiving Systems

Physical Intelligence

Robotic Materials

Social Foundations of Computation

Research Groups

Autonomous Vision

Autonomous Learning

Bioinspired Autonomous Miniature Robots

Dynamic Locomotion

Embodied Vision

Human Aspects of Machine Learning

Intelligent Control Systems

Learning and Dynamical Systems

Locomotion in Biorobotic and Somatic Systems

Micro, Nano, and Molecular Systems

Movement Generation and Control

Neural Capture and Synthesis

Physics for Inference and Optimization

Organizational Leadership and Diversity

Probabilistic Learning Group

Topics

Robot Learning

Conference Paper

2022

Autonomous Learning

Robotics

AI

Career

Award

Learning and Dynamical Systems Conference Paper Adaptive Decision-Making with Constraints and Dependent Losses: Performance Guarantees and Applications to Online and Nonlinear Identification Muehlebach, M. IFAC World Congress, 2023 (Published) URL BibTeX
Empirical Inference Article Altered brain dynamics index levels of arousal in complete locked-in syndrome Zilio, F., Gomez-Pilar, J., Chaudhary, U., Fogel, S., Fomina, T., Synofzik, M., Schöls, L., Cao, S., Zhang, J., Huang, Z., Birbaumer, N., Northoff, G. Communications Biology, 6(1), 2023 (Published) DOI BibTeX
Dynamic Locomotion Unpublished An Open-Source Modular Treadmill for Dynamic Force Measurement with Load Dependant Range Adjustment Sarvestani, A., Ruppert, F., Badri-Spröwitz, A. 2023 (Submitted)
Ground reaction force sensing is one of the key components of gait analysis in legged locomotion research. To measure continuous force data during locomotion, we present a novel compound instrumented treadmill design. The treadmill is 1.7 m long, with a natural frequency of 170 Hz and an adjustable range that can be used for humans and small robots alike. Here, we present the treadmill’s design methodology and characterize it in its natural frequency, noise behavior and real-life performance. Additionally, we apply an ISO 376 norm conform calibration procedure for all spatial force directions and center of pressure position. We achieve a force accuracy of ≤ 5.6 N for the ground reaction forces and ≤ 13 mm in center of pressure position.
arXiv DOI URL BibTeX
Materials Article Atomic-resolution observations of silver segregation in a [111] tilt grain boundary in copper Langeohl, L., Brink, T., Richter, G., Dehm, G., Liebscher, C. H. Physical Review B, 107(13):134112, American Physical Society, Woodbury, NY, 2023 (Published) DOI BibTeX
Empirical Inference Article Avoiding Shortcut-Learning by Mutual Information Minimization in Deep Learning-Based Image Processing Fay, L., Cobos, E., Yang, B., Gatidis, S., Küstner, T. IEEE Access, 11:64070-64086, 2023 (Published) DOI BibTeX
Movement Generation and Control Empirical Inference Autonomous Motion Article BiConMP: A Nonlinear Model Predictive Control Framework for Whole Body Motion Planning Meduri, A., Shah, P., Viereck, J., Khadiv, M., Havoutis, I., Righetti, L. IEEE Transactions on Robotics, 39(2):905-922, IEEE, 2023 (Published) DOI URL BibTeX
Physical Intelligence Article Bio-inspired rotary flight of light-driven nanocomposite films Wang, D., Chen, Z., Li, M., Hou, Z., Zhan, C., Zheng, Q., Wang, D., Wang, X., Cheng, M., Hu, W., Sitti, M., others, 2023 DOI BibTeX
Robotic Materials Article Biodegradable Electrohydraulic Actuators for Sustainable Soft Robots Rumley, E. H., Preninger, D., Shagan-Shomron, A., Rothemund, P., Hartmann, F., Baumgartner, M., Kellaris, N., Stojanovic, A., Yoder, Z., Karrer, B., Keplinger, C., Kaltenbrunner, M. Science Advances, 9(12), 2023, Ellen H. Rumley and David Preninger were co-first authors, and Christoph Keplinger and Martin Kaltenbrunner were shared corresponding authors. (Published)
Combating environmental pollution demands a focus on sustainability, in particular from rapidly advancing technologies that are poised to be ubiquitous in modern societies. Among these, soft robotics promises to replace conventional rigid machines for applications requiring adaptability and dexterity. For key components of soft robots, such as soft actuators, it is thus important to explore sustainable options like bioderived and biodegradable materials. We introduce systematically determined compatible materials systems for the creation of fully biodegradable, high-performance electrohydraulic soft actuators, based on various biodegradable polymer films, ester-based liquid dielectric, and NaCl-infused gelatin hydrogel. We demonstrate that these biodegradable actuators reliably operate up to high electric fields of 200 V/μm, show performance comparable to nonbiodegradable counterparts, and survive more than 100,000 actuation cycles. Furthermore, we build a robotic gripper based on biodegradable soft actuators that is readily compatible with commercial robot arms, encouraging wider use of biodegradable materials systems in soft robotics.
YouTube video DOI BibTeX
Physical Intelligence Article Bioinspired microstructured adhesives with facile and fast switchability for part manipulation in dry and wet conditions Pang, C., Kim, J., Wu, Y., Wang, M. Y., Yu, H., Sitti, M. Advanced Functional Materials, 33(38):2303116, 2023 (Published) DOI BibTeX
Physical Intelligence Article Bioinspired rotary flight of light-driven composite films Wang, D., Chen, Z., Li, M., Hou, Z., Zhan, C., Zheng, Q., Wang, D., Wang, X., Cheng, M., Hu, W., others, Nature Communications, 14(1):5070, 2023 DOI BibTeX
Physical Intelligence Article Bioinspired self-assembled colloidal collectives drifting in three dimensions underwater Sun, M., Yang, S., Jiang, J., Jiang, S., Sitti, M., Zhang, L. Science Advances, 9(45):eadj4201, 2023 DOI BibTeX
Micro, Nano, and Molecular Systems Article Biomolecular actuators for genetically selective acoustic manipulation of cells Wu, D., Baresch, D., Cook, C., Ma, Z., Duan, M., Malounda, D., Maresca, D., Abundo, M. P., Lee, J., Shivaei, S., Mittelstein, D. R., Qiu, T., Fischer, P., Shapiro, M. G. Science Advances, 9(8):eadd9186, AAAS, Washington, 2023 (Published) DOI URL BibTeX
Empirical Inference Miscellaneous Borges und die Künstliche Intelligenz Bottou, L., Schölkopf, B. 2023, published in Frankfurter Allgemeine Zeitung, 18 December 2023, Nr. 294 (Published) PDF BibTeX
Physical Intelligence Article Broad-Wavelength Light-Driven High-Speed Hybrid Crystal Actuators Actuated Inside Tissue-Like Phantoms Kim, D. W., Hagiwara, Y., Hasebe, S., Dogan, N. O., Zhang, M., Asahi, T., Koshima, H., Sitti, M. Advanced Functional Materials, 33(47):2305916, 2023 DOI BibTeX
Physical Intelligence Patent Carrier, use of a carrier, method of activating a carrier and method of making a carrier Drotlef, D., Sitti, M., Amjadi, M. 2023, US Patent App. 16/500,442 BibTeX
Physical Intelligence Article Cholesteric Liquid Crystal Polymeric Coatings for Colorful Artificial Muscles and Motile Humidity Sensor Skin Integrated with Magnetic Composites Feng, W., Pal, A., Wang, T., Ren, Z., Yan, Y., Lu, Y., Yang, H., Sitti, M. Advanced Functional Materials, 33(23), 2023 (Published) DOI URL BibTeX
Embodied Vision Conference Paper Context-Conditional Navigation with a Learning-Based Terrain- and Robot-Aware Dynamics Model Guttikonda, S., Achterhold, J., Li, H., Boedecker, J., Stueckler, J. In Proceedings of the European Conference on Mobile Robots (ECMR), 2023 (Published)
In autonomous navigation settings, several quantities can be subject to variations. Terrain properties such as friction coefficients may vary over time depending on the location of the robot. Also, the dynamics of the robot may change due to, e.g., different payloads, changing the system's mass, or wear and tear, changing actuator gains or joint friction. An autonomous agent should thus be able to adapt to such variations. In this paper, we develop a novel probabilistic, terrain- and robot-aware forward dynamics model, termed TRADYN, which is able to adapt to the above-mentioned variations. It builds on recent advances in meta-learning forward dynamics models based on Neural Processes. We evaluate our method in a simulated 2D navigation setting with a unicycle-like robot and different terrain layouts with spatially varying friction coefficients. In our experiments, the proposed model exhibits lower prediction error for the task of long-horizon trajectory prediction, compared to non-adaptive ablation models. We also evaluate our model on the downstream task of navigation planning, which demonstrates improved performance in planning control-efficient paths by taking robot and terrain properties into account.
preprint code DOI URL BibTeX
Neural Capture and Synthesis Conference Paper DINER: Depth-aware Image-based Neural Radiance Fields Prinzler, M., Hilliges, O., Thies, J. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), CVPR, 2023 (Accepted)
We present Depth-aware Image-based NEural Radiance fields (DINER). Given a sparse set of RGB input views, we predict depth and feature maps to guide the reconstruction of a volumetric scene representation that allows us to render 3D objects under novel views. Specifically, we propose novel techniques to incorporate depth information into feature fusion and efficient scene sampling. In comparison to the previous state of the art, DINER achieves higher synthesis quality and can process input views with greater disparity. This allows us to capture scenes more completely without changing capturing hardware requirements and ultimately enables larger viewpoint changes during novel view synthesis. We evaluate our method by synthesizing novel views, both for human heads and for general objects, and observe significantly improved qualitative results and increased perceptual metrics compared to the previous state of the art. The code is publicly available for research purposes.
Video Code Arxiv URL BibTeX
Learning and Dynamical Systems Conference Paper Data-Efficient Online Learning of Ball Placement in Robot Table Tennis Tobuschat, P., Ma, H., Büchler, D., Schölkopf, B., Muehlebach, M. EEE/RSJ International Conference on Intelligent Robots and Systems, 2023 (Published) URL BibTeX
Physical Intelligence Article Deployable Soft Origami Modular Robotic Arm With Variable Stiffness Using Facet Buckling Park, M., Kim, W., Yu, S., Cho, J., Kang, W., Byun, J., Jeong, U., Cho, K. IEEE Robotics and Automation Letters, 8(2):864-871, 2023 (Published) DOI BibTeX
Physical Intelligence Article Design and build of small-scale magnetic soft-bodied robots with multimodal locomotion Ren, Z., Sitti, M. Nature Protocols, 19(2):441-486, 2023 DOI BibTeX
Physical Intelligence Article Designing Covalent Organic Framework-based Light-driven Microswimmers towards Intraocular Theranostic Applications Sridhar, V., Yildiz, E., Rodrı́guez-Camargo, A., Lyu, X., Yao, L., Wrede, P., Aghakhani, A., Akolpoglu, M. B., Podjaski, F., Lotsch, B. V., Sitti, M. Advanced Materials, 35(25), 2023 (Published)
While micromachines with tailored functionalities enable therapeutic applications in biological environments, their controlled motion and targeted drug delivery in biological media require sophisticated designs for practical applications. Covalent organic frameworks (COFs), a new generation of crystalline and nanoporous polymers, offer new perspectives for light-driven microswimmers in heterogeneous biological environments including intraocular fluids, thus setting the stage for biomedical applications such as retinal drug delivery. Two different types of COFs, uniformly spherical TABP-PDA-COF sub-micrometer particles and texturally nanoporous, micrometer-sized TpAzo-COF particles are described and compared as light-driven microrobots. They can be used as highly efficient visible-light-driven drug carriers in aqueous ionic and cellular media. Their absorption ranging down to red light enables phototaxis even in deeper and viscous biological media, while the organic nature of COFs ensures their biocompatibility. Their inherently porous structures with ≈2.6 and ≈3.4 nm pores, and large surface areas allow for targeted and efficient drug loading even for insoluble drugs, which can be released on demand. Additionally, indocyanine green (ICG) dye loading in the pores enables photoacoustic imaging, optical coherence tomography, and hyperthermia in operando conditions. This real-time visualization of the drug-loaded COF microswimmers enables unique insights into the action of photoactive porous drug carriers for therapeutic applications.
DOI BibTeX
Modern Magnetic Systems Article Direct observation of Néel-type skyrmions and domain walls in a ferrimagnetic DyCo3 thin film Luo, C., Chen, K., Ukleev, V., Wintz, S., Weigand, M., Abrudan, R., Prokes, K., Radu, F. Communications Physics, 6:2018, Nature Publishing Group, London, 2023 (Published) DOI BibTeX
Modern Magnetic Systems Article Direct observation of propagating spin waves in the 2D van der Waals ferromagnet Fe5GeTe2 Schulz, F., Litzius, K., Powalla, L., Birch, M. T., Gallardo, R. A., Satheesh, S., Weigand, M., Scholz, T., Lotsch, B. V., Schütz, G., Burghard, M., Wintz, S. Nano Letters, 23(22):10126-10131, American Chemical Society, Washington, DC, 2023 DOI BibTeX
Autonomous Learning Article Discovering causal relations and equations from data Camps-Valls, G., Gerhardus, A., Ninad, U., Varando, G., Martius, G., Balaguer-Ballester, E., Vinuesa, R., Diaz, E., Zanna, L., Runge, J. Physics Reports, 1044:1-68, 2023
Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws, and principles that are invariant, robust, and causal has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing the world and, when possible, performing interventions on the system under study. With the advent of big data and data-driven methods, the fields of causal and equation discovery have developed and accelerated progress in computer science, physics, statistics, philosophy, and many applied fields. This paper reviews the concepts, methods, and relevant works on causal and equation discovery in the broad field of physics and outlines the most important challenges and promising future lines of research. We also provide a taxonomy for data-driven causal and equation discovery, point out connections, and showcase comprehensive case studies in Earth and climate sciences, fluid dynamics and mechanics, and the neurosciences. This review demonstrates that discovering fundamental laws and causal relations by observing natural phenomena is revolutionised with the efficient exploitation of observational data and simulations, modern machine learning algorithms and the combination with domain knowledge. Exciting times are ahead with many challenges and opportunities to improve our understanding of complex systems.
DOI BibTeX
Physical Intelligence Patent Dry adhesives and methods for making dry adhesives M Sitti, M. M. B. A. 2023, US Patent 11,773,298, 2023 BibTeX
Modern Magnetic Systems Article Ferromagnetic order controlled by the magnetic interface of LaNiO3/ La2/3Ca1/3MnO3 superlattices Soltan, S., Macke, S., Ilse, S. E., Pennycook, T., Zhang, Z. L., Christiani, G., Benckiser, E., Schütz, G., Goering, E. Scientific Reports, 13:3847, Nature Publishing Group, London, UK, 2023 (Published) DOI BibTeX
Physical Intelligence Article Hydrogel muscles powering reconfigurable micro-metastructures with wide-spectrum programmability Zhang, M., Pal, A., Zheng, Z., Gardi, G., Yildiz, E., Sitti, M. Nature Materials, 22(10):1243-1252, 2023 DOI BibTeX
Modern Magnetic Systems Article Imaging of short-wavelength spin waves in a nanometer-thick YIG/Co bilayer Talapatra, A., Qin, H., Schulz, F., Yao, L., Flajsman, L., Weigand, M., Wintz, S., van Dijken, S. Applied Physics Letters, 122(20):202404, American Institute of Physics, Melville, NY, 2023 (Published) DOI BibTeX
Empirical Inference Article Information theoretic measures of causal influences during transient neural events Shao, K., Logothetis, N. K., Besserve, M. Frontiers in Network Physiology, 3, 2023 (Published) DOI URL BibTeX
Physical Intelligence Article Integrated mechanologics for autonomous soft machines Pal, A., Byun, J., Ko, J., Sitti, M. Bulletin of the American Physical Society, 2023 BibTeX
Autonomous Learning Article Interpretable Symbolic Regression for Data Science: Analysis of the 2022 Competition Franca, F. D., Virgolin, M., Kommenda, M., Majumder, M., Cranmer, M., Espada, G., Ingelse, L., Fonseca, A., Landajuela, M., Petersen, B., Glatt, R., Mundhenk, N., Lee, C., Hochhalter, J., Randall, D., Kamienny, P., Zhang, H., Dick, G., Simon, A., Burlacu, B., et al. arXiv, 2023 URL BibTeX
Embodied Vision Conference Paper Learning-based Relational Object Matching Across Views Elich, C., Armeni, I., Oswald, M. R., Pollefeys, M., Stueckler, J. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023 (Published)
Intelligent robots require object-level scene understanding to reason about possible tasks and interactions with the environment. Moreover, many perception tasks such as scene reconstruction, image retrieval, or place recognition can benefit from reasoning on the level of objects. While keypoint-based matching can yield strong results for finding correspondences for images with small to medium view point changes, for large view point changes, matching semantically on the object-level becomes advantageous. In this paper, we propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network. We demonstrate our approach in a large variety of views on realistically rendered synthetic images. Our approach compares favorably to previous state-of-the-art object-level matching approaches and achieves improved performance over a pure keypoint-based approach for large view-point changes.
preprint code DOI URL BibTeX
Physical Intelligence Article Liquid Metal Microdroplet-Initiated Ultra-Fast Polymerization of a Stimuli-Responsive Hydrogel Composite Zhang, J., Liao, J., Liu, Z., Zhang, R., Sitti, M. Advanced Functional Materials, 2023 DOI BibTeX
Physical Intelligence Article MRI-powered Magnetic Miniature Capsule Robot with HIFU-controlled On-demand Drug Delivery Tiryaki, M. E., Dogangun, F., Dayan, C. B., Wrede, P., Sitti, M. arXiv preprint arXiv:2301.07197, 2023 DOI BibTeX
Physical Intelligence Article Machine Learning-Based Shear Optimal Adhesive Microstructures with Experimental Validation Dayan, C. B., Son, D., Aghakhani, A., Wu, Y., Demir, S. O., Sitti, M. Small, 2304437, 2023 DOI BibTeX
Physical Intelligence Article Magnetic Putty as A Reconfigurable, Recyclable, And Accessible Soft Robotic Material Li, M., Pal, A., Byun, J., Gardi, G., Sitti, M. Advanced Materials, 2304825, 2023 DOI BibTeX
Modern Magnetic Systems Article Magnetic field sensing elements made of quasi-trapezoidal magnetoplasmonic crystals based on thin permalloy films Murzin, D., Belyaev, V. K., Groß, F., Gräfe, J., Perov, N., Komanicky, V., Rodionova, V. Journal of Magnetism and Magnetic Materials, 588, NH, Elsevier, Amsterdam, 2023 DOI BibTeX
Physical Intelligence Article Magnetic guidewire steering at ultrahigh magnetic fields Tiryaki, M. E., Elmacıoğlu, Y. G., Sitti, M. Science Advances, 9(17):eadg6438, 2023 DOI BibTeX
Modern Magnetic Systems Article Magnetic phase transition in dimer-based metal-organic frameworks through structural modulation Kim, S., Schütz, G., Son, K., Oh, H. Sustainable Materials and Technologies, 38, Elsevier, Amsterdam, 2023 DOI BibTeX
Physical Intelligence Patent Magnetic trap system and method of navigating a microscopic device Son, D., Ugurlu, M., Bluemer, P., Sitti, M. 2023, US Patent App. 17/871,598 BibTeX
Physical Intelligence Patent Method of fabricating a magnetic deformable machine and deformable 3D magnetic machine Zhang, J., Ren, Z., Hu, W., Sitti, M. 2023, US Patent App. 18/020,161 BibTeX
Empirical Inference Article Metrizing Weak Convergence with Maximum Mean Discrepancies Simon-Gabriel, C., Barp, A., Schölkopf, B., Mackey, L. Journal of Machine Learning Research, 24(184), 2023 (Published)
This paper characterizes the maximum mean discrepancies (MMD) that metrize the weak convergence of probability measures for a wide class of kernels. More precisely, we prove that, on a locally compact, non-compact, Hausdorff space, the MMD of a bounded continuous Borel measurable kernel k, whose RKHS-functions vanish at infinity (i.e., Hk ⊂ C0), metrizes the weak convergence of probability measures if and only if k is continuous and integrally strictly positive definite (∫ s.p.d.) over all signed, finite, regular Borel measures. We also correct a prior result of Simon-Gabriel and Schölkopf (JMLR 2018, Thm. 12) by showing that there exist both bounded continuous ∫ s.p.d. kernels that do not metrize weak convergence and bounded continuous non-∫ s.p.d. kernels that do metrize it
arXiv URL BibTeX
Physical Intelligence Article Micro-and nanofabrication of dynamic hydrogels with multichannel information Zhang, M., Lee, Y., Zheng, Z., Khan, M. T. A., Lyu, X., Byun, J., Giessen, H., Sitti, M. Nature Communications, 14(1):8208, 2023 DOI BibTeX
Physical Intelligence Patent Microfibers with mushroom-shaped tips for optimal adhesion Sitti, M., Aksak, B. 2023, US Patent 11,613,674 BibTeX
Physical Intelligence Article Microrobotic Locomotion in Blood Vessels: A Computational Study on the Performance of Surface Microrollers in the Cardiovascular System Bozuyuk, U., Ozturk, H., Sitti, M. Advanced Intelligent Systems, 5(9):2300099, 2023 DOI BibTeX
Empirical Inference Article Mimicking Tumor Cell Heterogeneity of Colorectal Cancer in a Patient-derived Organoid-Fibroblast Model Atanasova, V. S., de Jesus Cardona, C., Hejret, V., Tiefenbacher, A., Mair, T., Tran, L., Pfneissl, J., Draganić, K., Binder, C., Kabiljo, J., Clement, J., Woeran, K., Neudert, B., Wohlhaupter, S., Haase, A., Domazet, S., Hengstschläger, M., Mitterhauser, M., Müllauer, L., Tichý, B., et al. Cellular and molecular gastroenterology and hepatology, 15(6):1391-1419, 2023 (Published) DOI BibTeX
Dynamic Locomotion Article Muscle Preflex Response to Perturbations in locomotion: In-vitro experiments and simulations with realistic boundary conditions Araz, M., Weidner, S., Izzi, F., Badri-Spröwitz, A., Siebert, T., Haeufle, A. D. F. B. Frontiers in Bioengineering and Biotechnology, 11, 2023 (Published)
Neuromuscular control loops feature substantial communication delays, but mammals run robustly even in the most adverse conditions. In-vivo experiments and computer simulation results suggest that muscles’ preflex—an immediate mechanical response to a perturbation—could be the critical contributor. Muscle preflexes act within a few milliseconds, an order of magnitude faster than neural reflexes. Their short-lasting activity makes mechanical preflexes hard to quantify in-vivo. Muscle models, on the other hand, require further improvement of their prediction accuracy during the non-standard conditions of perturbed locomotion. Additionally, muscles mechanically adapt by increased damping force. Our study aims to quantify the mechanical preflex work and test its mechanical force adaptation. We performed in-vitro experiments with biological muscle fibers under physiological boundary conditions, which we determined in computer simulations of perturbed hopping. Our findings show that muscles initially resist impacts with a stereotypical sti↵ness response—identified as short-range sti↵ness—regardless of the exact perturbation condition. We then observe a velocity adaptation to the force related to the amount of perturbation. The main contributor to the preflex work adaptation is not the force di↵erence but the muscle fiber stretch di↵erence. We find that both muscle sti↵ness and damping are activity-dependent properties. These results indicate that neural control could tune the preflex properties of muscles in expectation of ground conditions leading to previously inexplicable neuromuscular adaptation speeds.
DOI URL BibTeX
Modern Magnetic Systems Article Nanoporous adsorbents for hydrogen storage Hirscher, M., Zhang, L., Oh, H. Applied Physics A, 129(2), Springer-Verlag Heidelberg, Heidelberg, 2023 DOI BibTeX
Empirical Inference Book Chapter Natural Language Processing for Policymaking Jin, Z., Mihalcea, R. In Handbook of Computational Social Science for Policy, 141-162, 7, (Editors: Bertoni, E. and Fontana, M. and Gabrielli, L. and Signorelli, S. and Vespe, M.), Springer International Publishing, 2023 (Published) DOI BibTeX