Institute Homepage
Institute Homepage DE Sign In

Publications

DEPARTMENTS

Emperical Interference

Haptic Intelligence

Modern Magnetic Systems

Perceiving Systems

Physical Intelligence

Robotic Materials

Social Foundations of Computation

Research Groups

Autonomous Vision

Autonomous Learning

Bioinspired Autonomous Miniature Robots

Dynamic Locomotion

Embodied Vision

Human Aspects of Machine Learning

Intelligent Control Systems

Learning and Dynamical Systems

Locomotion in Biorobotic and Somatic Systems

Micro, Nano, and Molecular Systems

Movement Generation and Control

Neural Capture and Synthesis

Physics for Inference and Optimization

Organizational Leadership and Diversity

Probabilistic Learning Group

Topics

Robot Learning

Conference Paper

2022

Autonomous Learning

Robotics

AI

Career

Award

Physical Intelligence Patent Microfibers with mushroom-shaped tips for optimal adhesion Sitti, M., Aksak, B. 2020, US Patent 10,689,549 URL BibTeX
Physical Intelligence Article Microribbons composed of directionally self-assembled nanoflakes as highly stretchable ionic neural electrodes Zhang, M., Guo, R., Chen, K., Wang, Y., Niu, J., Guo, Y., Zhang, Y., Yin, Z., Xia, K., Zhou, B., Wang, H., He, W., Liu, J., Sitti, M., Zhang, Y. Proceedings of the National Academy of Sciences, 117(26):14667-14675, 2020 DOI URL BibTeX
Physical Intelligence Article Microscale polarization color pixels from liquid crystal elastomers Guo, Y., Shahsavan, H., Sitti, M. Advanced Optical Materials, 8(17):1902098, Wiley-VCH, 2020 (Published) DOI BibTeX
Physical Intelligence Article Modal analysis of finite-size piezoelectric metamaterial plates Aghakhani, A., Murat Gozum, M., Basdogan, I. Journal of Physics D: Applied Physics, 53(50):505304, 2020 DOI URL BibTeX
Empirical Inference Conference Paper Model-Based Quality-Diversity Search for Efficient Robot Learning Keller, L., Tanneberg, D., Stark, S., Peters, J. International Conference on Intelligent Robots and Systems (IROS), 9675-9680, IEEE, 2020 (Published) DOI BibTeX
Locomotion in Biorobotic and Somatic Systems Conference Paper Modulation of Cranio-Caudal mass distribution facilitates obstacle traversal in a cursorial biorobotic model Siddall, R. J. D., Jusufi, A. In Integrative and Comparative Biology, 60(Supplement 1):E214-E214, Society for Integrative and Comparative Biology Annual Meeting (SICB Annual Meeting 2020), 2020 DOI URL BibTeX
Article Motion Planning Explorer: Visualizing Local Minima Using a Local-Minima Tree Orthey, A., Frész, B., Toussaint, M. IEEE Robotics and Automation Letters, 5(2):346-353, IEEE, New York, NY, 2020 DOI BibTeX
Empirical Inference Article Multi-Channel Interactive Reinforcement Learning for Sequential Tasks Koert, D., Kircher, M., Salikutluk, V., D’Eramo, C., Peters, J. Frontiers in Robotics and AI, 7(September), 2020 (Published) DOI BibTeX
Empirical Inference Article Multi-Sensor Next-Best-View Planning as Matroid-Constrained Submodular Maximization Lauri, M., Pajarinen, J., Peters, J., Frintrop, S. IEEE Robotics and Automation Letters, 5(4):5323-5300, 2020 (Published) DOI BibTeX
Physical Intelligence Bioinspired Autonomous Miniature Robots Article Multifunctional magnetic soft composites: a review Wu, S., Hu, W., Ze, Q., Sitti, M., Zhao, R. Multifunctional Materials, 3(4):042003, 2020 DOI URL BibTeX
Physical Intelligence Article Multifunctional surface microrollers for targeted cargo delivery in physiological blood flow Alapan, Y., Bozuyuk, U., Erkoc, P., Karacakol, A. C., Sitti, M. Science Robotics, 5(42):eaba5726, 2020 (Published) DOI URL BibTeX
Physical Intelligence Article Multiwavelength-steerable visible-light-driven magnetic CoO-TiO2 microswimmers Sridhar, V., Park, B., Guo, S., van Aken, P. A., Sitti, M. ACS Applied Materials & Interfaces, 12(21):24149-24155, 2020 DOI BibTeX
Article Muscles Reduce Neuronal Information Load: Quantification of Control Effort in Biological vs. Robotic Pointing and Walking Haeufle, D. F. B., Wochner, I., Holzmüller, D., Driess, D., Günther, M., Schmitt, S. Frontiers in Robotics and AI, 7, Frontiers Media, Lausanne, 2020 DOI BibTeX
Physical Intelligence Article Nanoerythrosome-functionalized biohybrid microswimmers Buss, N., Yasa, O., Alapan, Y., Akolpoglu, M. B., Sitti, M. APL Bioengineering, 4(2):026103, 2020 (Published) DOI BibTeX
Modern Magnetic Systems Article Nanoimaging of ultrashort magnon emission by ferromagnetic grating couplers at GHz frequencies Baumgaertl, K., Gräfe, J., Che, P., Mucchietto, A., Förster, J., Träger, N., Bechtel, M., Weigand, M., Schütz, G., Grundler, D. Nano Letters, 20(10):7281-7286, American Chemical Society, Washington, DC, 2020 DOI BibTeX
Modern Magnetic Systems Article Noncollinear antiferromagnetic order in the buckled honeycomb lattice of magnetoelectric Co4Ta2O9 determined by single-crystal neutron diffraction Choi, S., Oh, D. G., Gutmann, M. J., Pan, S., Kim, G., Son, K., Kim, J., Lee, N., Cheong, S., Choi, Y. J., Kiryukhin, V. Physical Review B, 102(21), American Physical Society, Woodbury, NY, 2020 DOI BibTeX
Embodied Vision Empirical Inference Article Numerical Quadrature for Probabilistic Policy Search Vinogradska, J., Bischoff, B., Achterhold, J., Koller, T., Peters, J. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(1):164-175, 2020 (Published) DOI BibTeX
Theory of Inhomogeneous Condensed Matter Article Numerical simulations of self-diffusiophoretic colloids at fluid interfaces Peter, T., Malgaretti, P., Rivas, N., Scagliarini, A., Harting, J., Dietrich, S. Soft Matter, 16(14):3536-3547, Royal Society of Chemistry, Cambridge, UK, 2020 DOI BibTeX
Modern Magnetic Systems Article Observation of compact ferrimagnetic skyrmions in DyCo3 fim Chen, K., Lott, D., Philippi-Kobs, A., Weigand, M., Luo, C., Radu, F. Nanoscale, 12(35):18137-18143, Royal Society of Chemistry, Cambridge, UK, 2020 DOI BibTeX
Autonomous Vision Conference Paper On Joint Estimation of Pose, Geometry and svBRDF from a Handheld Scanner Schmitt, C., Donne, S., Riegler, G., Koltun, V., Geiger, A. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2020
We propose a novel formulation for joint recovery of camera pose, object geometry and spatially-varying BRDF. The input to our approach is a sequence of RGB-D images captured by a mobile, hand-held scanner that actively illuminates the scene with point light sources. Compared to previous works that jointly estimate geometry and materials from a hand-held scanner, we formulate this problem using a single objective function that can be minimized using off-the-shelf gradient-based solvers. By integrating material clustering as a differentiable operation into the optimization process, we avoid pre-processing heuristics and demonstrate that our model is able to determine the correct number of specular materials independently. We provide a study on the importance of each component in our formulation and on the requirements of the initial geometry. We show that optimizing over the poses is crucial for accurately recovering fine details and that our approach naturally results in a semantically meaningful material segmentation.
pdf Project Page Slides Video Poster BibTeX
Human Aspects of Machine Learning Conference Paper On the Compressibility of Affinely Singular Random Vectors Charusaie, M., Rini, S., Amini, A. On the Compressibility of Affinely Singular Random Vectors, 2240-2245, 2020 DOI BibTeX
Intelligent Control Systems Article Online learning with stability guarantees: A memory-based warm starting for real-time MPC Schwenkel, L., Gharbi, M., Trimpe, S., Ebenbauer, C. Automatica, 122:109247, 2020 (Published) DOI BibTeX
Physical Intelligence Article Paper-based microchip electrophoresis for point-of-care hemoglobin testing Hasan, M. N., Fraiwan, A., An, R., Alapan, Y., Ung, R., Akkus, A., Xu, J. Z., Rezac, A. J., Kocmich, N. J., Creary, M. S., Oginni, T., Olanipekun, G. M., Hassan-Hanga, F., Jibir, B. W., Gambo, S., Verma, A. K., Bharti, P. K., Riolueang, S., Ngimhung, T., Suksangpleng, T., et al. The Analyst, 145(7):2525-2542, 2020
We present a versatile, mass-producible, paper-based microchip electrophoresis platform that enables rapid, affordable, decentralized hemoglobin testing at the point-of-care. , Nearly 7\% of the world's population live with a hemoglobin variant. Hemoglobins S, C, and E are the most common and significant hemoglobin variants worldwide. Sickle cell disease, caused by hemoglobin S, is highly prevalent in sub-Saharan Africa and in tribal populations of Central India. Hemoglobin C is common in West Africa, and hemoglobin E is common in Southeast Asia. Screening for significant hemoglobin disorders is not currently feasible in many low-income countries with the high disease burden. Lack of early diagnosis leads to preventable high morbidity and mortality in children born with hemoglobin variants in low-resource settings. Here, we describe HemeChip, the first miniaturized, paper-based, microchip electrophoresis platform for identifying the most common hemoglobin variants easily and affordably at the point-of-care in low-resource settings. HemeChip test works with a drop of blood. HemeChip system guides the user step-by-step through the test procedure with animated on-screen instructions. Hemoglobin identification and quantification is automatically performed, and hemoglobin types and percentages are displayed in an easily understandable, objective way. We show the feasibility and high accuracy of HemeChip via testing 768 subjects by clinical sites in the United States, Central India, sub-Saharan Africa, and Southeast Asia. Validation studies include hemoglobin E testing in Bangkok, Thailand, and hemoglobin S testing in Chhattisgarh, India, and in Kano, Nigeria, where the sickle cell disease burden is the highest in the world. Tests were performed by local users, including healthcare workers and clinical laboratory personnel. Study design, methods, and results are presented according to the Standards for Reporting Diagnostic Accuracy (STARD). HemeChip correctly identified all subjects with hemoglobin S, C, and E variants with 100\% sensitivity, and displayed an overall diagnostic accuracy of 98.4\% in comparison to reference standard methods. HemeChip is a versatile, mass-producible microchip electrophoresis platform that addresses a major unmet need of decentralized hemoglobin analysis in resource-limited settings.
DOI URL BibTeX
Theory of Inhomogeneous Condensed Matter Article Particles, string and interface in the three-dimensional Ising model Delfino, G., Selke, W., Squarcini, A. Nuclear Physics (Amsterdam) B, 958:115139, North-Holland, Amsterdam, 2020 DOI BibTeX
Theory of Inhomogeneous Condensed Matter Article Passive advection of fractional Brownian motion by random layered flows Squarcini, A., Marinari, E., Oshanin, G. New Journal of Physics, 22(5):053052, IOP Publishing, Bristol, 2020 DOI BibTeX
Physical Intelligence Article Peeling the layers back: examining the roles of capillary adhesion and material softening on gecko and gecko-inspired synthetic adhesive performance in variable temperature and humidity Mitchell, C. T., Drotlef, D., Dayan, C. B., Sitti, M., Stark, A. Y. Integrative and Comparative Biology, 60:E163-E163, 2020 DOI BibTeX
Empirical Inference Conference Paper Physically constrained causal noise models for high-contrast imaging of exoplanets Gebhard, T. D., Bonse, M. J., Quanz, S. P., Schölkopf, B. Machine Learning and the Physical Sciences - Workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS), 2020 (Published) arXiv BibTeX
Embodied Vision Conference Paper Planning from Images with Deep Latent Gaussian Process Dynamics Bosch, N., Achterhold, J., Leal-Taixe, L., Stückler, J. Proceedings of the 2nd Conference on Learning for Dynamics and Control (L4DC), 120:640-650, Proceedings of Machine Learning Research (PMLR), (Editors: Alexandre M. Bayen and Ali Jadbabaie and George Pappas and Pablo A. Parrilo and Benjamin Recht and Claire Tomlin and Melanie Zeilinger), 2020, preprint arXiv:2005.03770 (Published) Ppreprint Project page Code poster URL BibTeX
Materials Article Plastic Forming of Metals at the Nanoscale: Interdiffusion-Induced Bending of Bimetallic Nanowhiskers Qi, Y., Richter, G., Suadiye, E., Kalina, M., Rabkin, E. ACS Nano, 14(9):11691-11699, American Chemical Society, Washington, DC, 2020 (Published) DOI BibTeX
Empirical Inference Article Plucking Motions for Tea Harvesting Robots Using Probabilistic Movement Primitives Motokura, K., Takahashi, M., Ewerton, M., Peters, J. IEEE Robotics and Automation Letters, 5(2):3275-3282, 2020 (Published) DOI BibTeX
Physical Intelligence Article Poly(ethylene glycol)–poly(beta-amino ester)-based nanoparticles for suicide gene therapy enhance brain penetration and extend survival in a preclinical human glioblastoma orthotopic xenograft model Kim, J., Mondal, S. K., Tzeng, S. Y., Rui, Y., Al-kharboosh, R., Kozielski, K. K., Bhargav, A. G., Garcia, C. A., Quiñones-Hinojosa, A., Green, J. J. ACS Biomaterials Science & Engineering, 6(5):2943-2955, 2020 DOI URL BibTeX
Autonomous Motion Conference Paper Prediction of Human Full-Body Movements with Motion Optimization and Recurrent Neural Networks Kratzer, P., Toussaint, M., Mainprice, J. In 2020 IEEE International Conference on Robotics and Automation (ICRA 2020), 1792-1798, IEEE, Piscataway, NJ, IEEE International Conference on Robotics and Automation (ICRA 2020), 2020 (Published) DOI BibTeX
Probabilistic Learning Group Article Preliminary Results on Different Text Processing Tasks Using Encoder-Decoder Networks and the Causal Feature Extractor Javaloy, A. G. G. Applied Sciences, 10(17):5772, 2020 (Published) DOI BibTeX
Empirical Inference Article Probabilistic Approach to Physical Object Disentangling Pajarinen, J., Arenz, O., Peters, J., Neumann, G. IEEE Robotics and Automation Letters, 5(4):5510-5517, 2020 (Published) DOI BibTeX
Micro, Nano, and Molecular Systems Patent Propeller and method in which a propeller is set into motion Qui, T., Fischer, P. (US20200031010A1), 2020
A method where a propeller is set into locomotion relative to a medium at least partially surrounding the propeller. An actuator induces a rotation of the propeller relative to the medium and about a rotational axis of the propeller, and the propeller converts its rotational movement into locomotion relative to the medium. The aspect ratio of at least one cross-section of the propeller is three or more. Also a helical or modifiedly helical propeller for converting rotational movement of the propeller into locomotion of the propeller relative to a medium at least partially surrounding the propeller, where the aspect ratio of at least one cross section of the propeller is three or more. And a method of producing a propeller, including the step of providing a plate extending along the helical axis, where the aspect ratio of at least one cross section of the plate is three or more.
BibTeX
Physical Intelligence Article Pros and cons: magnetic versus optical microrobots Sitti, M., Wiersma, D. S. Advanced Materials, 32(20):1906766, 2020
Mobile microrobotics has emerged as a new robotics field within the last decade to create untethered tiny robots that can access and operate in unprecedented, dangerous, or hard‐to‐reach small spaces noninvasively toward disruptive medical, biotechnology, desktop manufacturing, environmental remediation, and other potential applications. Magnetic and optical actuation methods are the most widely used actuation methods in mobile microrobotics currently, in addition to acoustic and biological (cell‐driven) actuation approaches. The pros and cons of these actuation methods are reported here, depending on the given context. They can both enable long‐range, fast, and precise actuation of single or a large number of microrobots in diverse environments. Magnetic actuation has unique potential for medical applications of microrobots inside nontransparent tissues at high penetration depths, while optical actuation is suitable for more biotechnology, lab‐/organ‐on‐a‐chip, and desktop manufacturing types of applications with much less surface penetration depth requirements or with transparent environments. Combining both methods in new robot designs can have a strong potential of combining the pros of both methods. There is still much progress needed in both actuation methods to realize the potential disruptive applications of mobile microrobots in real‐world conditions.
DOI BibTeX
Modern Magnetic Systems Article Ptychographic imaging and micromagnetic modeling of thermal melting of nanoscale magnetic domains in antidot lattices Gräfe, J., Skripnik, M., Dieterle, G., Haering, F., Weigand, M., Bykova, I., Träger, N., Stoll, H., Tyliszczak, T., Vine, D., Ziemann, P., Wiedwald, U., Shapiro, D., Nowak, U., Schütz, G., Goering, E. J. AIP Advances, 10(12), American Institute of Physics, Melville, NY, USA, 2020 DOI BibTeX
Modern Magnetic Systems Article Quantification of competing magnetic states and switching pathways in curved nanowires by direct dynamic imaging Schönke, D., Reeve, R. M., Stoll, H., Kläui, M. ACS Nano, 14(10):13324-13332, American Chemical Society, Washington, DC, 2020 DOI BibTeX
Materials Article Rational strain engineering in delafossite oxides for highly efficient hydrogen evolution catalysis in acidic media Podjaski, F., Weber, D., Zhang, S. Y., Diehl, L., Eger, R., Duppel, V., Alarcon-Llado, E., Richter, G., Haase, F., Morral, A. F. I., Scheu, C., Lotsch, B. V. Nature Catalysis, 3(1):55-63, 2020 DOI BibTeX
Physical Intelligence Article Recent advances in plant nanobionics and nanobiosensors for toxicology applications Ansari, M. H., Lavhale, S., Kalunke, R. M., Srivastava, P. L., Pandit, V., Gade, S., Yadav, S., Laux, P., Luch, A., Gemmati, D., others, Current Nanoscience, 16(1):27-41, 2020 DOI BibTeX
Physical Intelligence Article Reconfigurable multifunctional ferrofluid droplet robots Fan, X., Dong, X., Karacakol, A. C., Xie, H., Sitti, M. Proceedings of the National Academy of Sciences, 117(45):27916-27926, 2020 DOI URL BibTeX
Modern Magnetic Systems Article Reconfigurable submicrometer spin-wave majority gate with electrical transducers Talmelli, G., Devolder, T., Träger, N., Förster, J., Wintz, S., Weigand, M., Stoll, H., Heyns, M., Schütz, G., Radu, I. P., Gräfe, J., Ciubotaru, F., Adelmann, C. Science Advances, 6(51), AAAS, Washington, 2020 DOI BibTeX
Theory of Inhomogeneous Condensed Matter Article Reconfiguring confined magnetic colloids with tunable fluid transport behavior Sheng, Z., Zhang, M., Liu, J., Malgaretti, P., Li, J., Wang, S., Lv, W., Zhang, R., Fan, Y., Zhang, Y., Chen, X., Hou, X. National Science Review, 8(5):nwaa301, Oxford University Press, Oxford, 2020 DOI BibTeX
Physical Intelligence Article Reprogrammable shape morphing of magnetic soft machines Alapan, Y., Karacakol, A. C., Guzelhan, S. N., Isik, I., Sitti, M. Science Advances, 6(38):eabc6414, 2020 DOI URL BibTeX
Modern Magnetic Systems Article Research trend of metal-organic frameworks for magnetic refrigeration materials application Kim, S., Son, K., Oh, H. Korean Journal of Materials Research, 30(3):136-141, Materials Society of Korea, Seoul, 2020 DOI BibTeX
Proceedings Robotics: Science and System XVI RSS Foundation, Corvalis, OR, 2020 BibTeX
Conference Paper Robust Task and Motion Planning for Long-Horizon Architectural Construction Planning Hartmann, V. N., Oguz, O. S., Driess, D., Toussaint, M., Menges, A. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020), 6886-6893, IEEE, Piscataway, NJ, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020), 2020 (Published) DOI BibTeX
Empirical Inference Article SCIM: universal single-cell matching with unpaired feature sets Stark, S. G., Ficek, J., Locatello, F., Bonilla, X., Chevrier, S., Singer, F., Tumor Profiler Consortium, , Rätsch, G., Lehmann, K. Bioinformatics, 36:i919-i927, 2020, Supplement\textunderscore2 (Published) DOI BibTeX
Intelligent Control Systems Autonomous Motion Article Safe and Fast Tracking on a Robot Manipulator: Robust MPC and Neural Network Control Nubert, J., Koehler, J., Berenz, V., Allgower, F., Trimpe, S. IEEE Robotics and Automation Letters, 5(2):3050-3057, 2020 (Published)
Fast feedback control and safety guarantees are essential in modern robotics. We present an approach that achieves both by combining novel robust model predictive control (MPC) with function approximation via (deep) neural networks (NNs). The result is a new approach for complex tasks with nonlinear, uncertain, and constrained dynamics as are common in robotics. Specifically, we leverage recent results in MPC research to propose a new robust setpoint tracking MPC algorithm, which achieves reliable and safe tracking of a dynamic setpoint while guaranteeing stability and constraint satisfaction. The presented robust MPC scheme constitutes a one-layer approach that unifies the often separated planning and control layers, by directly computing the control command based on a reference and possibly obstacle positions. As a separate contribution, we show how the computation time of the MPC can be drastically reduced by approximating the MPC law with a NN controller. The NN is trained and validated from offline samples of the MPC, yielding statistical guarantees, and used in lieu thereof at run time. Our experiments on a state-of-the-art robot manipulator are the first to show that both the proposed robust and approximate MPC schemes scale to real-world robotic systems.
arXiv PDF DOI BibTeX
Empirical Inference Conference Paper Sample-Efficient Reinforcement Learning via Counterfactual-Based Data Augmentation Lu, C., Huang, B., Wang, K., Hernández-Lobato, J. M., Zhang, K., Schölkopf, B. Offline Reinforcement Learning - Workshop at the 34th Conference on Neural Information Processing Systems (NeurIPS), 2020 (Published) arXiv URL BibTeX