Header logo is

321 results (View BibTeX file of all listed publications)

2024

no image
Language Models Can Reduce Asymmetry in Information Markets

Rahaman, N., Weiss, M., Wüthrich, M., Bengio, Y., Li, E., Pal, C., Schölkopf, B.

arXiv:2403.14443, March 2024, Published as: Redesigning Information Markets in the Era of Language Models, Conference on Language Modeling (COLM) (techreport)

Abstract
This work addresses the buyer's inspection paradox for information markets. The paradox is that buyers need to access information to determine its value, while sellers need to limit access to prevent theft. To study this, we introduce an open-source simulated digital marketplace where intelligent agents, powered by language models, buy and sell information on behalf of external participants. The central mechanism enabling this marketplace is the agents' dual capabilities: they not only have the capacity to assess the quality of privileged information but also come equipped with the ability to forget. This ability to induce amnesia allows vendors to grant temporary access to proprietary information, significantly reducing the risk of unauthorized retention while enabling agents to accurately gauge the information's relevance to specific queries or tasks. To perform well, agents must make rational decisions, strategically explore the marketplace through generated sub-queries, and synthesize answers from purchased information. Concretely, our experiments (a) uncover biases in language models leading to irrational behavior and evaluate techniques to mitigate these biases, (b) investigate how price affects demand in the context of informational goods, and (c) show that inspection and higher budgets both lead to higher quality outcomes.

ei

link (url) [BibTex]

2024

ei Rahaman, N., Weiss, M., Wüthrich, M., Bengio, Y., Li, E., Pal, C., Schölkopf, B. Language Models Can Reduce Asymmetry in Information Markets arXiv:2403.14443, March 2024, Published as: Redesigning Information Markets in the Era of Language Models, Conference on Language Modeling (COLM) (techreport)

link (url) [BibTex]

no image
Learning a Terrain- and Robot-Aware Dynamics Model for Autonomous Mobile Robot Navigation

Achterhold, J., Guttikonda, S., Kreber, J. U., Li, H., Stueckler, J.

CoRR abs/2409.11452, 2024, Preprint submitted to Robotics and Autonomous Systems Journal. https://arxiv.org/abs/2409.11452 (techreport) Submitted

Abstract
Mobile robots should be capable of planning cost-efficient paths for autonomous navigation. Typically, the terrain and robot properties are subject to variations. For instance, properties of the terrain such as friction may vary across different locations. Also, properties of the robot may change such as payloads or wear and tear, e.g., causing changing actuator gains or joint friction. Autonomous navigation approaches should thus be able to adapt to such variations. In this article, we propose a novel approach for learning a probabilistic, terrain- and robot-aware forward dynamics model (TRADYN) which can adapt to such variations and demonstrate its use for navigation. Our learning approach extends recent advances in meta-learning forward dynamics models based on Neural Processes for mobile robot navigation. We evaluate our method in simulation for 2D navigation of a robot with uni-cycle dynamics with varying properties on terrain with spatially varying friction coefficients. In our experiments, we demonstrate that TRADYN has lower prediction error over long time horizons than model ablations which do not adapt to robot or terrain variations. We also evaluate our model for navigation planning in a model-predictive control framework and under various sources of noise. We demonstrate that our approach yields improved performance in planning control-efficient paths by taking robot and terrain properties into account.

ev

preprint [BibTex]

ev Achterhold, J., Guttikonda, S., Kreber, J. U., Li, H., Stueckler, J. Learning a Terrain- and Robot-Aware Dynamics Model for Autonomous Mobile Robot Navigation CoRR abs/2409.11452, 2024, Preprint submitted to Robotics and Autonomous Systems Journal. https://arxiv.org/abs/2409.11452 (techreport) Submitted

preprint [BibTex]

no image
A Pontryagin Perspective on Reinforcement Learning

Eberhard, O., Vernade, C., Muehlebach, M.

Max Planck Institute for Intelligent Systems, 2024 (techreport)

lds

link (url) [BibTex]

lds Eberhard, O., Vernade, C., Muehlebach, M. A Pontryagin Perspective on Reinforcement Learning Max Planck Institute for Intelligent Systems, 2024 (techreport)

link (url) [BibTex]

no image
Distributed Event-Based Learning via ADMM

Er, D., Trimpe, S., Muehlebach, M.

Max Planck Institute for Intelligent Systems, 2024 (techreport)

lds

link (url) [BibTex]

lds Er, D., Trimpe, S., Muehlebach, M. Distributed Event-Based Learning via ADMM Max Planck Institute for Intelligent Systems, 2024 (techreport)

link (url) [BibTex]

no image
Incremental Few-Shot Adaptation for Non-Prehensile Object Manipulation using Parallelizable Physics Simulators

Baumeister, F., Mack, L., Stueckler, J.

CoRR abs/2409.13228, CoRR, 2024, Submitted to IEEE International Conference on Robotics and Automation (ICRA) 2025 (techreport) Submitted

Abstract
Few-shot adaptation is an important capability for intelligent robots that perform tasks in open-world settings such as everyday environments or flexible production. In this paper, we propose a novel approach for non-prehensile manipulation which iteratively adapts a physics-based dynamics model for model-predictive control. We adapt the parameters of the model incrementally with a few examples of robot-object interactions. This is achieved by sampling-based optimization of the parameters using a parallelizable rigid-body physics simulation as dynamic world model. In turn, the optimized dynamics model can be used for model-predictive control using efficient sampling-based optimization. We evaluate our few-shot adaptation approach in several object pushing experiments in simulation and with a real robot.

ev

preprint supplemental video link (url) [BibTex]

ev Baumeister, F., Mack, L., Stueckler, J. Incremental Few-Shot Adaptation for Non-Prehensile Object Manipulation using Parallelizable Physics Simulators CoRR abs/2409.13228, CoRR, 2024, Submitted to IEEE International Conference on Robotics and Automation (ICRA) 2025 (techreport) Submitted

preprint supplemental video link (url) [BibTex]

2023

Fairness in Machine Learning: Limitations and Opportunities
Fairness in Machine Learning: Limitations and Opportunities

Barocas, S., Hardt, M., Narayanan, A.

MIT Press, December 2023 (book)

Abstract
An introduction to the intellectual foundations and practical utility of the recent work on fairness and machine learning. Fairness and Machine Learning introduces advanced undergraduate and graduate students to the intellectual foundations of this recently emergent field, drawing on a diverse range of disciplinary perspectives to identify the opportunities and hazards of automated decision-making. It surveys the risks in many applications of machine learning and provides a review of an emerging set of proposed solutions, showing how even well-intentioned applications may give rise to objectionable results. It covers the statistical and causal measures used to evaluate the fairness of machine learning models as well as the procedural and substantive aspects of decision-making that are core to debates about fairness, including a review of legal and philosophical perspectives on discrimination. This incisive textbook prepares students of machine learning to do quantitative work on fairness while reflecting critically on its foundations and its practical utility.• Introduces the technical and normative foundations of fairness in automated decision-making• Covers the formal and computational methods for characterizing and addressing problems• Provides a critical assessment of their intellectual foundations and practical utility• Features rich pedagogy and extensive instructor resources

sf

link (url) [BibTex]

2023

sf Barocas, S., Hardt, M., Narayanan, A. Fairness in Machine Learning: Limitations and Opportunities MIT Press, December 2023 (book)

link (url) [BibTex]

An Open-Source Modular Treadmill for Dynamic Force Measurement with Load Dependant Range Adjustment
An Open-Source Modular Treadmill for Dynamic Force Measurement with Load Dependant Range Adjustment

Sarvestani, A., Ruppert, F., Badri-Spröwitz, A.

2023 (unpublished) Submitted

Abstract
Ground reaction force sensing is one of the key components of gait analysis in legged locomotion research. To measure continuous force data during locomotion, we present a novel compound instrumented treadmill design. The treadmill is 1.7 m long, with a natural frequency of 170 Hz and an adjustable range that can be used for humans and small robots alike. Here, we present the treadmill’s design methodology and characterize it in its natural frequency, noise behavior and real-life performance. Additionally, we apply an ISO 376 norm conform calibration procedure for all spatial force directions and center of pressure position. We achieve a force accuracy of ≤ 5.6 N for the ground reaction forces and ≤ 13 mm in center of pressure position.

dlg

arXiv link (url) DOI [BibTex]

no image
Natural Language Processing for Policymaking

Jin, Z., Mihalcea, R.

In Handbook of Computational Social Science for Policy, pages: 141-162, 7, (Editors: Bertoni, E. and Fontana, M. and Gabrielli, L. and Signorelli, S. and Vespe, M.), Springer International Publishing, 2023 (inbook)

ei

DOI [BibTex]

ei Jin, Z., Mihalcea, R. Natural Language Processing for Policymaking In Handbook of Computational Social Science for Policy, pages: 141-162, 7, (Editors: Bertoni, E. and Fontana, M. and Gabrielli, L. and Signorelli, S. and Vespe, M.), Springer International Publishing, 2023 (inbook)

DOI [BibTex]

Synchronizing Machine Learning Algorithms, Realtime Robotic Control and Simulated Environment with o80
Synchronizing Machine Learning Algorithms, Realtime Robotic Control and Simulated Environment with o80

Berenz, V., Widmaier, F., Guist, S., Schölkopf, B., Büchler, D.

Robot Software Architectures Workshop (RSA) 2023, ICRA, 2023 (techreport)

Abstract
Robotic applications require the integration of various modalities, encompassing perception, control of real robots and possibly the control of simulated environments. While the state-of-the-art robotic software solutions such as ROS 2 provide most of the required features, flexible synchronization between algorithms, data streams and control loops can be tedious. o80 is a versatile C++ framework for robotics which provides a shared memory model and a command framework for real-time critical systems. It enables expert users to set up complex robotic systems and generate Python bindings for scientists. o80's unique feature is its flexible synchronization between processes, including the traditional blocking commands and the novel ``bursting mode'', which allows user code to control the execution of the lower process control loop. This makes it particularly useful for setups that mix real and simulated environments.

ei

arxiv poster link (url) [BibTex]

2022

Magnetic Micro-/Nanopropellers for Biomedicine
Magnetic Micro-/Nanopropellers for Biomedicine

Qiu, T., Jeong, M., Goyal, R., Kadiri, V., Sachs, J., Fischer, P.

In Field-Driven Micro and Nanorobots for Biology and Medicine, pages: 389-410, 16, (Editors: Sun, Y. and Wang, X. and Yu, J.), Springer, Cham, 2022 (inbook)

Abstract
In nature, many bacteria swim by rotating their helical flagella. A particularly promising class of artificial micro- and nano-robots mimic this propeller-like propulsion mechanism to move through fluids and tissues for applications in minimally-invasive medicine. Several fundamental challenges have to be overcome in order to build micro-machines that move similar to bacteria for in vivo applications. Here, we review recent advances of magnetically-powered micro-/nano-propellers. Four important aspects of the propellers – the geometrical shape, the fabrication method, the generation of magnetic fields for actuation, and the choice of biocompatible magnetic materials – are highlighted. First, the fundamental requirements are elucidated that arise due to hydrodynamics at low Reynolds (Re) number. We discuss the role that the propellers’ shape and symmetry play in realizing effective propulsion at low Re. Second, the additive nano-fabrication method Glancing Angle Deposition is discussed as a versatile technique to quickly grow large numbers of designer nano-helices. Third, systems to generate rotating magnetic fields via permanent magnets or electromagnetic coils are presented. And finally, the biocompatibility of the magnetic materials is discussed. Iron-platinum is highlighted due to its biocompatibility and its superior magnetic properties, which is promising for targeted delivery, minimally-invasive magnetic nano-devices and biomedical applications.

pf

link (url) DOI [BibTex]

2022

pf Qiu, T., Jeong, M., Goyal, R., Kadiri, V., Sachs, J., Fischer, P. Magnetic Micro-/Nanopropellers for Biomedicine In Field-Driven Micro and Nanorobots for Biology and Medicine, pages: 389-410, 16, (Editors: Sun, Y. and Wang, X. and Yu, J.), Springer, Cham, 2022 (inbook)

link (url) DOI [BibTex]

no image
Life Improvement Science

Lieder, F., Prentice, M.

In Encyclopedia of Quality of Life and Well-Being Research, Springer, November 2022 (inbook)

re

DOI [BibTex]

re Lieder, F., Prentice, M. Life Improvement Science In Encyclopedia of Quality of Life and Well-Being Research, Springer, November 2022 (inbook)

DOI [BibTex]

no image
Patterns, Predictions, and Actions: Foundations of Machine Learning

Hardt, M., Recht, B.

Princeton University Press, August 2022 (book)

Abstract
An authoritative, up-to-date graduate textbook on machine learning that highlights its historical context and societal impacts Patterns, Predictions, and Actions introduces graduate students to the essentials of machine learning while offering invaluable perspective on its history and social implications. Beginning with the foundations of decision making, Moritz Hardt and Benjamin Recht explain how representation, optimization, and generalization are the constituents of supervised learning. They go on to provide self-contained discussions of causality, the practice of causal inference, sequential decision making, and reinforcement learning, equipping readers with the concepts and tools they need to assess the consequences that may arise from acting on statistical decisions. Provides a modern introduction to machine learning, showing how data patterns support predictions and consequential actions Pays special attention to societal impacts and fairness in decision making Traces the development of machine learning from its origins to today Features a novel chapter on machine learning benchmarks and datasets Invites readers from all backgrounds, requiring some experience with probability, calculus, and linear algebra An essential textbook for students and a guide for researchers

sf

link (url) [BibTex]

sf Hardt, M., Recht, B. Patterns, Predictions, and Actions: Foundations of Machine Learning Princeton University Press, August 2022 (book)

link (url) [BibTex]

no image
Observability Analysis of Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models

Li, H., Stueckler, J.

abs/2204.06651, CoRR/arxiv, 2022 (techreport)

Abstract
In this paper, we analyze the observability of the visual-inertial odometry (VIO) using stereo cameras with a velocity-control based kinematic motion model. Previous work shows that in general case the global position and yaw are unobservable in VIO system, additionally the roll and pitch become also unobservable if there is no rotation. We prove that by integrating a planar motion constraint roll and pitch become observable. We also show that the parameters of the motion model are observable.

ev

link (url) [BibTex]

no image
Causal Models for Dynamical Systems

Peters, J., Bauer, S., Pfister, N.

In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 671-690, 1, Association for Computing Machinery, 2022 (inbook)

ei

arXiv DOI [BibTex]

ei Peters, J., Bauer, S., Pfister, N. Causal Models for Dynamical Systems In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 671-690, 1, Association for Computing Machinery, 2022 (inbook)

arXiv DOI [BibTex]

no image
Towards Causal Algorithmic Recourse

Karimi, A. H., von Kügelgen, J., Schölkopf, B., Valera, I.

In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 139-166, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

ei plg

DOI [BibTex]

ei plg Karimi, A. H., von Kügelgen, J., Schölkopf, B., Valera, I. Towards Causal Algorithmic Recourse In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 139-166, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

DOI [BibTex]

no image
CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations

Salewski, L., Koepke, A. S., Lensch, H. P. A., Akata, Z.

In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 69-88, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

ei

DOI [BibTex]

ei Salewski, L., Koepke, A. S., Lensch, H. P. A., Akata, Z. CLEVR-X: A Visual Reasoning Dataset for Natural Language Explanations In xxAI - Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pages: 69-88, (Editors: Holzinger, Andreas and Goebel, Randy and Fong, Ruth and Moon, Taesup and Müller, Klaus-Robert and Samek, Wojciech), Springer International Publishing, 2022 (inbook)

DOI [BibTex]

no image
Causality for Machine Learning

Schölkopf, B.

In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 765-804, 1, Association for Computing Machinery, New York, NY, USA, 2022 (inbook)

ei

arXiv DOI [BibTex]

ei Schölkopf, B. Causality for Machine Learning In Probabilistic and Causal Inference: The Works of Judea Pearl, pages: 765-804, 1, Association for Computing Machinery, New York, NY, USA, 2022 (inbook)

arXiv DOI [BibTex]

2021

no image
Electriflow: Augmenting Books With Tangible Animation Using Soft Electrohydraulic Actuators

Purnendu, , Novack, S., Acome, E., Alistar, M., Keplinger, C., Gross, M. D., Bruns, C., Leithinger, D.

In ACM SIGGRAPH 2021 Labs, pages: 1-2, Association for Computing Machinery, SIGGRAPH 2021, August 2021 (inbook)

Abstract
We present Electriflow: a method of augmenting books with tangible animation employing soft electrohydraulic actuators. These actuators are compact, silent and fast in operation, and can be fabricated with commodity materials. They generate an immediate hydraulic force upon electrostatic activation without an external fluid supply source, enabling a simple and self-contained design. Electriflow actuators produce an immediate shape transition from flat to folded state which enabled their seamless integration into books. For the Emerging Technologies exhibit, we will demonstrate the prototype of a book augmented with the capability of tangible animation.

rm

Supplemental Material link (url) DOI [BibTex]

2021

rm Purnendu, , Novack, S., Acome, E., Alistar, M., Keplinger, C., Gross, M. D., Bruns, C., Leithinger, D. Electriflow: Augmenting Books With Tangible Animation Using Soft Electrohydraulic Actuators In ACM SIGGRAPH 2021 Labs, pages: 1-2, Association for Computing Machinery, SIGGRAPH 2021, August 2021 (inbook)

Supplemental Material link (url) DOI [BibTex]

Toward a Science of Effective Well-Doing
Toward a Science of Effective Well-Doing

Lieder, F., Prentice, M., Corwin-Renner, E.

May 2021 (techreport)

Abstract
Well-doing, broadly construed, encompasses acting and thinking in ways that contribute to humanity’s flourishing in the long run. This often takes the form of setting a prosocial goal and pursuing it over an extended period of time. To set and pursue goals in a way that is extremely beneficial for humanity (effective well-doing), people often have to employ critical thinking and far-sighted, rational decision-making in the service of the greater good. To promote effective well-doing, we need to better understand its determinants and psychological mechanisms, as well as the barriers to effective well-doing and how they can be overcome. In this article, we introduce a taxonomy of different forms of well-doing and introduce a conceptual model of the cognitive mechanisms of effective well-doing. We view effective well-doing as the upper end of a moral continuum whose lower half comprises behaviors that are harmful to humanity (ill-doing), and we argue that the capacity for effective well-doing has to be developed through personal growth (e.g., learning how to pursue goals effectively). Research on these phenomena has so far been scattered across numerous disconnected literatures from multiple disciplines. To bring these communities together, we call for the establishment of a transdisciplinary research field focussed on understanding and promoting effective well-doing and personal growth as well as understanding and reducing ill-doing. We define this research field in terms of its goals and questions. We review what is already known about these questions in different disciplines and argue that laying the scientific foundation for promoting effective well-doing is one of the most valuable contributions that the behavioral sciences can make in the 21st century.

re

Preprint Project Page [BibTex]

no image
Reinforcement Learning Algorithms: Analysis and Applications

Belousov, B., H., A., Klink, P., Parisi, S., Peters, J.

883, Studies in Computational Intelligence, Springer International Publishing, 2021 (book)

ei

DOI [BibTex]

ei Belousov, B., H., A., Klink, P., Parisi, S., Peters, J. Reinforcement Learning Algorithms: Analysis and Applications 883, Studies in Computational Intelligence, Springer International Publishing, 2021 (book)

DOI [BibTex]

2020

Optimal To-Do List Gamification
Optimal To-Do List Gamification

Stojcheski, J., Felso, V., Lieder, F.

ArXiv Preprint, 2020 (techreport)

Abstract
What should I work on first? What can wait until later? Which projects should I prioritize and which tasks are not worth my time? These are challenging questions that many people face every day. People’s intuitive strategy is to prioritize their immediate experience over the long-term consequences. This leads to procrastination and the neglect of important long-term projects in favor of seemingly urgent tasks that are less important. Optimal gamification strives to help people overcome these problems by incentivizing each task by a number of points that communicates how valuable it is in the long-run. Unfortunately, computing the optimal number of points with standard dynamic programming methods quickly becomes intractable as the number of a person’s projects and the number of tasks required by each project increase. Here, we introduce and evaluate a scalable method for identifying which tasks are most important in the long run and incentivizing each task according to its long-term value. Our method makes it possible to create to-do list gamification apps that can handle the size and complexity of people’s to-do lists in the real world.

re

link (url) DOI Project Page [BibTex]

no image
TUM Flyers: Vision-Based MAV Navigation for Systematic Inspection of Structures

Usenko, V., Stumberg, L. V., Stückler, J., Cremers, D.

In Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users: The Experience of the European Robotics Challenges, 136, pages: 189-209, Springer Tracts in Advanced Robotics, Springer International Publishing, 2020 (inbook)

ev

link (url) DOI [BibTex]

ev Usenko, V., Stumberg, L. V., Stückler, J., Cremers, D. TUM Flyers: Vision-Based MAV Navigation for Systematic Inspection of Structures In Bringing Innovative Robotic Technologies from Research Labs to Industrial End-users: The Experience of the European Robotics Challenges, 136, pages: 189-209, Springer Tracts in Advanced Robotics, Springer International Publishing, 2020 (inbook)

link (url) DOI [BibTex]

no image
Adopting the Boundary Homogenization Approximation from Chemical Kinetics to Motile Chemically Active Particles

Popescu, M. N., Uspal, W. E.

In Chemical Kinetics, pages: 517-540, (Editors: Lindenberg, Katja and Metzler, Ralf and Oshanin, Gleb), World Scientific, New Jersey, NJ, 2020 (incollection)

icm

DOI [BibTex]

icm Popescu, M. N., Uspal, W. E. Adopting the Boundary Homogenization Approximation from Chemical Kinetics to Motile Chemically Active Particles In Chemical Kinetics, pages: 517-540, (Editors: Lindenberg, Katja and Metzler, Ralf and Oshanin, Gleb), World Scientific, New Jersey, NJ, 2020 (incollection)

DOI [BibTex]

no image
Soft Microrobots Based on Photoresponsive Materials

Palagi, S.

In Mechanically Responsive Materials for Soft Robotics, pages: 327-362, (Editors: Koshima, Hideko), Wiley-VCH, Weinheim, 2020 (incollection)

pf

DOI [BibTex]

pf Palagi, S. Soft Microrobots Based on Photoresponsive Materials In Mechanically Responsive Materials for Soft Robotics, pages: 327-362, (Editors: Koshima, Hideko), Wiley-VCH, Weinheim, 2020 (incollection)

DOI [BibTex]

Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art

Janai, J., Güney, F., Behl, A., Geiger, A.

12(1-3), Foundations and Trends® in Computer Graphics and Vision, now Publishers Inc., Hanover, MA, 2020 (book)

Abstract
Recent years have witnessed enormous progress in AI-related fields such as computer vision, machine learning, and autonomous vehicles. As with any rapidly growing field, it becomes increasingly difficult to stay up-to-date or enter the field as a beginner. While several survey papers on particular sub-problems have appeared, no comprehensive survey on problems, datasets, and methods in computer vision for autonomous vehicles has been published. This monograph attempts to narrow this gap by providing a survey on the state-of-the-art datasets and techniques. Our survey includes both the historically most relevant literature as well as the current state of the art on several specific topics, including recognition, reconstruction, motion estimation, tracking, scene understanding, and end-to-end learning for autonomous driving. Towards this goal, we analyze the performance of the state of the art on several challenging benchmarking datasets, including KITTI, MOT, and Cityscapes. Besides, we discuss open problems and current research challenges. To ease accessibility and accommodate missing references, we also provide a website that allows navigating topics as well as methods and provides additional information.

avg

pdf Project Page link DOI Project Page [BibTex]

avg Janai, J., Güney, F., Behl, A., Geiger, A. Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art 12(1-3), Foundations and Trends® in Computer Graphics and Vision, now Publishers Inc., Hanover, MA, 2020 (book)

pdf Project Page link DOI Project Page [BibTex]

2019

no image
Nanomagnetismus im Röntgenlicht

Schütz, G.

In Vielfältige Physik, pages: 173-182, Springer Spektrum, Berlin, Heidelberg, 2019 (incollection)

mms

DOI [BibTex]

2019

mms Schütz, G. Nanomagnetismus im Röntgenlicht In Vielfältige Physik, pages: 173-182, Springer Spektrum, Berlin, Heidelberg, 2019 (incollection)

DOI [BibTex]

no image
Prototyping Micro- and Nano-Optics with Focused Ion Beam Lithography

Keskinbora, K.

SL48, pages: 46, SPIE.Spotlight, SPIE Press, Bellingham, WA, 2019 (book)

mms

DOI [BibTex]

mms Keskinbora, K. Prototyping Micro- and Nano-Optics with Focused Ion Beam Lithography SL48, pages: 46, SPIE.Spotlight, SPIE Press, Bellingham, WA, 2019 (book)

DOI [BibTex]

Das Tier als Modell für Roboter, und Roboter als Modell für Tiere
Das Tier als Modell für Roboter, und Roboter als Modell für Tiere

Badri-Spröwitz, A.

In pages: 167-175, Springer, 2019 (incollection)

dlg

DOI [BibTex]

dlg Badri-Spröwitz, A. Das Tier als Modell für Roboter, und Roboter als Modell für Tiere In pages: 167-175, Springer, 2019 (incollection)

DOI [BibTex]

2018

Nanoscale robotic agents in biological fluids and tissues
Nanoscale robotic agents in biological fluids and tissues

Palagi, S., Walker, D. Q. T., Fischer, P.

In The Encyclopedia of Medical Robotics, 2, pages: 19-42, 2, (Editors: Desai, J. P. and Ferreira, A.), World Scientific, October 2018 (inbook)

Abstract
Nanorobots are untethered structures of sub-micron size that can be controlled in a non-trivial way. Such nanoscale robotic agents are envisioned to revolutionize medicine by enabling minimally invasive diagnostic and therapeutic procedures. To be useful, nanorobots must be operated in complex biological fluids and tissues, which are often difficult to penetrate. In this chapter, we first discuss potential medical applications of motile nanorobots. We briefly present the challenges related to swimming at such small scales and we survey the rheological properties of some biological fluids and tissues. We then review recent experimental results in the development of nanorobots and in particular their design, fabrication, actuation, and propulsion in complex biological fluids and tissues. Recent work shows that their nanoscale dimension is a clear asset for operation in biological tissues, since many biological tissues consist of networks of macromolecules that prevent the passage of larger micron-scale structures, but contain dynamic pores through which nanorobots can move.

pf

link (url) DOI [BibTex]

2018

pf Palagi, S., Walker, D. Q. T., Fischer, P. Nanoscale robotic agents in biological fluids and tissues In The Encyclopedia of Medical Robotics, 2, pages: 19-42, 2, (Editors: Desai, J. P. and Ferreira, A.), World Scientific, October 2018 (inbook)

link (url) DOI [BibTex]

no image
Haptics and Haptic Interfaces

Kuchenbecker, K. J.

In Encyclopedia of Robotics, (Editors: Marcelo H. Ang and Oussama Khatib and Bruno Siciliano), Springer, May 2018 (incollection)

Abstract
Haptics is an interdisciplinary field that seeks to both understand and engineer touch-based interaction. Although a wide range of systems and applications are being investigated, haptics researchers often concentrate on perception and manipulation through the human hand. A haptic interface is a mechatronic system that modulates the physical interaction between a human and his or her tangible surroundings. Haptic interfaces typically involve mechanical, electrical, and computational layers that work together to sense user motions or forces, quickly process these inputs with other information, and physically respond by actuating elements of the user’s surroundings, thereby enabling him or her to act on and feel a remote and/or virtual environment.

hi

DOI [BibTex]

hi Kuchenbecker, K. J. Haptics and Haptic Interfaces In Encyclopedia of Robotics, (Editors: Marcelo H. Ang and Oussama Khatib and Bruno Siciliano), Springer, May 2018 (incollection)

DOI [BibTex]

no image
Detailed Dense Inference with Convolutional Neural Networks via Discrete Wavelet Transform

Ma, L., Stueckler, J., Wu, T., Cremers, D.

arxiv, 2018, arXiv:1808.01834 (techreport)

ev

[BibTex]

ev Ma, L., Stueckler, J., Wu, T., Cremers, D. Detailed Dense Inference with Convolutional Neural Networks via Discrete Wavelet Transform arxiv, 2018, arXiv:1808.01834 (techreport)

[BibTex]

no image
Maschinelles Lernen: Entwicklung ohne Grenzen?

Schölkopf, B.

In Mit Optimismus in die Zukunft schauen. Künstliche Intelligenz - Chancen und Rahmenbedingungen, pages: 26-34, (Editors: Bender, G. and Herbrich, R. and Siebenhaar, K.), B&S Siebenhaar Verlag, 2018 (incollection)

ei

[BibTex]

ei Schölkopf, B. Maschinelles Lernen: Entwicklung ohne Grenzen? In Mit Optimismus in die Zukunft schauen. Künstliche Intelligenz - Chancen und Rahmenbedingungen, pages: 26-34, (Editors: Bender, G. and Herbrich, R. and Siebenhaar, K.), B&S Siebenhaar Verlag, 2018 (incollection)

[BibTex]

no image
Methods in Psychophysics

Wichmann, F. A., Jäkel, F.

In Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience, 5 (Methodology), 7, 4th, John Wiley & Sons, Inc., 2018 (inbook)

ei

[BibTex]

ei Wichmann, F. A., Jäkel, F. Methods in Psychophysics In Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience, 5 (Methodology), 7, 4th, John Wiley & Sons, Inc., 2018 (inbook)

[BibTex]

no image
Transfer Learning for BCIs

Jayaram, V., Fiebig, K., Peters, J., Grosse-Wentrup, M.

In Brain–Computer Interfaces Handbook, pages: 425-442, 22, (Editors: Chang S. Nam, Anton Nijholt and Fabien Lotte), CRC Press, 2018 (incollection)

ei

[BibTex]

ei Jayaram, V., Fiebig, K., Peters, J., Grosse-Wentrup, M. Transfer Learning for BCIs In Brain–Computer Interfaces Handbook, pages: 425-442, 22, (Editors: Chang S. Nam, Anton Nijholt and Fabien Lotte), CRC Press, 2018 (incollection)

[BibTex]

2017

Chapter 8 - Micro- and nanorobots in Newtonian and biological viscoelastic fluids
Chapter 8 - Micro- and nanorobots in Newtonian and biological viscoelastic fluids

Palagi, S., (Walker) Schamel, D., Qiu, T., Fischer, P.

In Microbiorobotics, pages: 133 - 162, 8, Micro and Nano Technologies, Second edition, Elsevier, Boston, March 2017 (incollection)

Abstract
Swimming microorganisms are a source of inspiration for small scale robots that are intended to operate in fluidic environments including complex biomedical fluids. Nature has devised swimming strategies that are effective at small scales and at low Reynolds number. These include the rotary corkscrew motion that, for instance, propels a flagellated bacterial cell, as well as the asymmetric beat of appendages that sperm cells or ciliated protozoa use to move through fluids. These mechanisms can overcome the reciprocity that governs the hydrodynamics at small scale. The complex molecular structure of biologically important fluids presents an additional challenge for the effective propulsion of microrobots. In this chapter it is shown how physical and chemical approaches are essential in realizing engineered abiotic micro- and nanorobots that can move in biomedically important environments. Interestingly, we also describe a microswimmer that is effective in biological viscoelastic fluids that does not have a natural analogue.

pf

link (url) DOI [BibTex]

2017

pf Palagi, S., (Walker) Schamel, D., Qiu, T., Fischer, P. Chapter 8 - Micro- and nanorobots in Newtonian and biological viscoelastic fluids In Microbiorobotics, pages: 133 - 162, 8, Micro and Nano Technologies, Second edition, Elsevier, Boston, March 2017 (incollection)

link (url) DOI [BibTex]

no image
Robot Learning

Peters, J., Lee, D., Kober, J., Nguyen-Tuong, D., Bagnell, J., Schaal, S.

In Springer Handbook of Robotics, pages: 357-394, 15, 2nd, (Editors: Siciliano, Bruno and Khatib, Oussama), Springer International Publishing, 2017 (inbook)

am ei

[BibTex]

am ei Peters, J., Lee, D., Kober, J., Nguyen-Tuong, D., Bagnell, J., Schaal, S. Robot Learning In Springer Handbook of Robotics, pages: 357-394, 15, 2nd, (Editors: Siciliano, Bruno and Khatib, Oussama), Springer International Publishing, 2017 (inbook)

[BibTex]

no image
Elements of Causal Inference - Foundations and Learning Algorithms

Peters, J., Janzing, D., Schölkopf, B.

Adaptive Computation and Machine Learning Series, The MIT Press, Cambridge, MA, USA, 2017 (book)

ei

PDF link (url) [BibTex]

ei Peters, J., Janzing, D., Schölkopf, B. Elements of Causal Inference - Foundations and Learning Algorithms Adaptive Computation and Machine Learning Series, The MIT Press, Cambridge, MA, USA, 2017 (book)

PDF link (url) [BibTex]

no image
Policy Gradient Methods

Peters, J., Bagnell, J.

In Encyclopedia of Machine Learning and Data Mining, pages: 982-985, 2nd, (Editors: Sammut, Claude and Webb, Geoffrey I.), Springer US, 2017 (inbook)

ei

link (url) [BibTex]

ei Peters, J., Bagnell, J. Policy Gradient Methods In Encyclopedia of Machine Learning and Data Mining, pages: 982-985, 2nd, (Editors: Sammut, Claude and Webb, Geoffrey I.), Springer US, 2017 (inbook)

link (url) [BibTex]

no image
Unsupervised clustering of EOG as a viable substitute for optical eye-tracking

Flad, N., Fomina, T., Bülthoff, H. H., Chuang, L. L.

In First Workshop on Eye Tracking and Visualization (ETVIS 2015), pages: 151-167, Mathematics and Visualization, (Editors: Burch, M., Chuang, L., Fisher, B., Schmidt, A., and Weiskopf, D.), Springer, 2017 (inbook)

ei

DOI [BibTex]

ei Flad, N., Fomina, T., Bülthoff, H. H., Chuang, L. L. Unsupervised clustering of EOG as a viable substitute for optical eye-tracking In First Workshop on Eye Tracking and Visualization (ETVIS 2015), pages: 151-167, Mathematics and Visualization, (Editors: Burch, M., Chuang, L., Fisher, B., Schmidt, A., and Weiskopf, D.), Springer, 2017 (inbook)

DOI [BibTex]

Mobile Microrobotics
Mobile Microrobotics

Sitti, M.

Mobile Microrobotics, The MIT Press, Cambridge, MA, 2017 (book)

Abstract
Progress in micro- and nano-scale science and technology has created a demand for new microsystems for high-impact applications in healthcare, biotechnology, manufacturing, and mobile sensor networks. The new robotics field of microrobotics has emerged to extend our interactions and explorations to sub-millimeter scales. This is the first textbook on micron-scale mobile robotics, introducing the fundamentals of design, analysis, fabrication, and control, and drawing on case studies of existing approaches. The book covers the scaling laws that can be used to determine the dominant forces and effects at the micron scale; models forces acting on microrobots, including surface forces, friction, and viscous drag; and describes such possible microfabrication techniques as photo-lithography, bulk micromachining, and deep reactive ion etching. It presents on-board and remote sensing methods, noting that remote sensors are currently more feasible; studies possible on-board microactuators; discusses self-propulsion methods that use self-generated local gradients and fields or biological cells in liquid environments; and describes remote microrobot actuation methods for use in limited spaces such as inside the human body. It covers possible on-board powering methods, indispensable in future medical and other applications; locomotion methods for robots on surfaces, in liquids, in air, and on fluid-air interfaces; and the challenges of microrobot localization and control, in particular multi-robot control methods for magnetic microrobots. Finally, the book addresses current and future applications, including noninvasive medical diagnosis and treatment, environmental remediation, and scientific tools.

pi

Mobile Microrobotics By Metin Sitti - Chapter 1 (PDF) link (url) [BibTex]

pi Sitti, M. Mobile Microrobotics Mobile Microrobotics, The MIT Press, Cambridge, MA, 2017 (book)

Mobile Microrobotics By Metin Sitti - Chapter 1 (PDF) link (url) [BibTex]

no image
Robot Learning

Peters, J., Tedrake, R., Roy, N., Morimoto, J.

In Encyclopedia of Machine Learning and Data Mining, pages: 1106-1109, 2nd, (Editors: Sammut, Claude and Webb, Geoffrey I.), Springer US, 2017 (inbook)

ei

DOI [BibTex]

ei Peters, J., Tedrake, R., Roy, N., Morimoto, J. Robot Learning In Encyclopedia of Machine Learning and Data Mining, pages: 1106-1109, 2nd, (Editors: Sammut, Claude and Webb, Geoffrey I.), Springer US, 2017 (inbook)

DOI [BibTex]

no image
Statistical Asymmetries Between Cause and Effect

Janzing, D.

In Time in Physics, pages: 129-139, Tutorials, Schools, and Workshops in the Mathematical Sciences, (Editors: Renner, Renato and Stupar, Sandra), Springer International Publishing, Cham, 2017 (inbook)

ei

link (url) DOI [BibTex]

ei Janzing, D. Statistical Asymmetries Between Cause and Effect In Time in Physics, pages: 129-139, Tutorials, Schools, and Workshops in the Mathematical Sciences, (Editors: Renner, Renato and Stupar, Sandra), Springer International Publishing, Cham, 2017 (inbook)

link (url) DOI [BibTex]

Decentralized Simultaneous Multi-target Exploration using a Connected Network of Multiple Robots
Decentralized Simultaneous Multi-target Exploration using a Connected Network of Multiple Robots

Nestmeyer, T., Robuffo Giordano, P., Bülthoff, H. H., Franchi, A.

In pages: 989-1011, Autonomous Robots, 2017 (incollection)

ps

[BibTex]

ps Nestmeyer, T., Robuffo Giordano, P., Bülthoff, H. H., Franchi, A. Decentralized Simultaneous Multi-target Exploration using a Connected Network of Multiple Robots In pages: 989-1011, Autonomous Robots, 2017 (incollection)

[BibTex]

no image
Momentum-Centered Control of Contact Interactions

Righetti, L., Herzog, A.

In Geometric and Numerical Foundations of Movements, 117, pages: 339-359, Springer Tracts in Advanced Robotics, Springer, Cham, 2017 (incollection)

mg

link (url) [BibTex]

mg Righetti, L., Herzog, A. Momentum-Centered Control of Contact Interactions In Geometric and Numerical Foundations of Movements, 117, pages: 339-359, Springer Tracts in Advanced Robotics, Springer, Cham, 2017 (incollection)

link (url) [BibTex]

2016

Implications of Action-Oriented Paradigm Shifts in Cognitive Science
Implications of Action-Oriented Paradigm Shifts in Cognitive Science

Dominey, P. F., Prescott, T. J., Bohg, J., Engel, A. K., Gallagher, S., Heed, T., Hoffmann, M., Knoblich, G., Prinz, W., Schwartz, A.

In The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science, 18, pages: 333-356, 20, Strüngmann Forum Reports, vol. 18, J. Lupp, series editor, (Editors: Andreas K. Engel and Karl J. Friston and Danica Kragic), The MIT Press, 18th Ernst Strüngmann Forum, May 2016 (incollection) In press

Abstract
An action-oriented perspective changes the role of an individual from a passive observer to an actively engaged agent interacting in a closed loop with the world as well as with others. Cognition exists to serve action within a landscape that contains both. This chapter surveys this landscape and addresses the status of the pragmatic turn. Its potential influence on science and the study of cognition are considered (including perception, social cognition, social interaction, sensorimotor entrainment, and language acquisition) and its impact on how neuroscience is studied is also investigated (with the notion that brains do not passively build models, but instead support the guidance of action). A review of its implications in robotics and engineering includes a discussion of the application of enactive control principles to couple action and perception in robotics as well as the conceptualization of system design in a more holistic, less modular manner. Practical applications that can impact the human condition are reviewed (e.g. educational applications, treatment possibilities for developmental and psychopathological disorders, the development of neural prostheses). All of this foreshadows the potential societal implications of the pragmatic turn. The chapter concludes that an action-oriented approach emphasizes a continuum of interaction between technical aspects of cognitive systems and robotics, biology, psychology, the social sciences, and the humanities, where the individual is part of a grounded cultural system.

am

The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science 18th Ernst Strüngmann Forum Bibliography Chapter link (url) [BibTex]

2016

am Dominey, P. F., Prescott, T. J., Bohg, J., Engel, A. K., Gallagher, S., Heed, T., Hoffmann, M., Knoblich, G., Prinz, W., Schwartz, A. Implications of Action-Oriented Paradigm Shifts in Cognitive Science In The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science, 18, pages: 333-356, 20, Strüngmann Forum Reports, vol. 18, J. Lupp, series editor, (Editors: Andreas K. Engel and Karl J. Friston and Danica Kragic), The MIT Press, 18th Ernst Strüngmann Forum, May 2016 (incollection) In press

The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science 18th Ernst Strüngmann Forum Bibliography Chapter link (url) [BibTex]

Learning Action-Perception Cycles in Robotics: A Question of Representations and Embodiment
Learning Action-Perception Cycles in Robotics: A Question of Representations and Embodiment

Bohg, J., Kragic, D.

In The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science, 18, pages: 309-320, 18, Strüngmann Forum Reports, vol. 18, J. Lupp, series editor, (Editors: Andreas K. Engel and Karl J. Friston and Danica Kragic), The MIT Press, 18th Ernst Strüngmann Forum, May 2016 (incollection) In press

Abstract
Since the 1950s, robotics research has sought to build a general-purpose agent capable of autonomous, open-ended interaction with realistic, unconstrained environments. Cognition is perceived to be at the core of this process, yet understanding has been challenged because cognition is referred to differently within and across research areas, and is not clearly defined. The classic robotics approach is decomposition into functional modules which perform planning, reasoning, and problem-solving or provide input to these mechanisms. Although advancements have been made and numerous success stories reported in specific niches, this systems-engineering approach has not succeeded in building such a cognitive agent. The emergence of an action-oriented paradigm offers a new approach: action and perception are no longer separable into functional modules but must be considered in a complete loop. This chapter reviews work on different mechanisms for action- perception learning and discusses the role of embodiment in the design of the underlying representations and learning. It discusses the evaluation of agents and suggests the development of a new embodied Turing Test. Appropriate scenarios need to be devised in addition to current competitions, so that abilities can be tested over long time periods.

am

18th Ernst Strüngmann Forum The Pragmatic Turn- Toward Action-Oriented Views in Cognitive Science Bibliography Chapter link (url) [BibTex]

am Bohg, J., Kragic, D. Learning Action-Perception Cycles in Robotics: A Question of Representations and Embodiment In The Pragmatic Turn - Toward Action-Oriented Views in Cognitive Science, 18, pages: 309-320, 18, Strüngmann Forum Reports, vol. 18, J. Lupp, series editor, (Editors: Andreas K. Engel and Karl J. Friston and Danica Kragic), The MIT Press, 18th Ernst Strüngmann Forum, May 2016 (incollection) In press

18th Ernst Strüngmann Forum The Pragmatic Turn- Toward Action-Oriented Views in Cognitive Science Bibliography Chapter link (url) [BibTex]