Institute Homepage
Institute Homepage EN Sign In

Publications

DEPARTMENTS

Emperical Interference

Haptic Intelligence

Modern Magnetic Systems

Perceiving Systems

Physical Intelligence

Robotic Materials

Social Foundations of Computation

Research Groups

Autonomous Vision

Autonomous Learning

Bioinspired Autonomous Miniature Robots

Dynamic Locomotion

Embodied Vision

Human Aspects of Machine Learning

Intelligent Control Systems

Learning and Dynamical Systems

Locomotion in Biorobotic and Somatic Systems

Micro, Nano, and Molecular Systems

Movement Generation and Control

Neural Capture and Synthesis

Physics for Inference and Optimization

Organizational Leadership and Diversity

Probabilistic Learning Group

Topics

Robot Learning

Conference Paper

2022

Autonomous Learning

Robotics

AI

Career

Award

Empirical Inference Proceedings Proceedings of the Second Workshop on NLP for Positive Impact (NLP4PI) Biester, L., Demszky, D., Jin, Z., Sachan, M., Tetreault, J., Wilson, S., Xiao, L., Zhao, J. Association for Computational Linguistics, December 2022 (Published) URL BibTeX
Proceedings Towards semi-automated pleural cavity access for pneumothorax in austere environments L’Orsa, R., Lama, S., Westwick, D., Sutherland, G., Kuchenbecker, K. J. International Astronautical Congress (IAC), September 2022 BibTeX
Empirical Inference Proceedings Proceedings of the First Conference on Causal Learning and Reasoning (CLeaR 2022) Schölkopf, B., Uhler, C., Zhang, K. 177, Proceedings of Machine Learning Research, PMLR, April 2022 (Published) URL BibTeX
Empirical Inference Proceedings Proceedings of the 1st Workshop on NLP for Positive Impact Field, A., Prabhumoye, S., Sap, M., Jin, Z., Zhao, J., Brockett, C. Association for Computational Linguistics, August 2021 (Published) URL BibTeX
Proceedings Jerk Control of Floating Base Systems With Contact-Stable Parameterized Force Feedback Gazar, A., Nava, G., Chavez, F. J. A., Pucci, D. IEEE Transactions on Robotics, (1)1-15, First, IEEE, February 2021 (Published)
Nonlinear controllers for floating base systems in contact with the environment are often framed as quadratic programming (QP) optimization problems. Common drawbacks of such QP-based controllers are: the control input often experiences discontinuities; no force feedback from force/torque (FT) sensors installed on the robot is taken into account. This article attempts to address these limitations using jerk-based control architectures. The proposed controllers assume the rate-of-change of the joint torques as control input, and exploit the system position, velocity, accelerations, and contact wrenches as measurable quantities. The key ingredient of the presented approach is a one-to-one correspondence between free variables and an inner approximation of the manifold defined by the contact stability constraints. More precisely, the proposed correspondence covers completely the contact stability manifold except for the socalled friction cone, for which there exists a unique correspondence for more than 90% of its elements. The correspondence allows us to transform the underlying constrained optimization problem into one that is unconstrained. Then, we propose a jerk control framework that exploits the proposed correspondence and uses FT measurements in the control loop. Furthermore, we present Lyapunov stable controllers for the system momentum in the jerk control framework. The approach is validated with simulations and experiments using the iCub humanoid robot.
DOI URL BibTeX
Intelligent Control Systems Autonomous Motion Proceedings Excursion Search for Constrained Bayesian Optimization under a Limited Budget of Failures Marco, A., Rohr, A. V., Baumann, D., Hernández-Lobato, J. M., Trimpe, S. 2020 (In revision)
When learning to ride a bike, a child falls down a number of times before achieving the first success. As falling down usually has only mild consequences, it can be seen as a tolerable failure in exchange for a faster learning process, as it provides rich information about an undesired behavior. In the context of Bayesian optimization under unknown constraints (BOC), typical strategies for safe learning explore conservatively and avoid failures by all means. On the other side of the spectrum, non conservative BOC algorithms that allow failing may fail an unbounded number of times before reaching the optimum. In this work, we propose a novel decision maker grounded in control theory that controls the amount of risk we allow in the search as a function of a given budget of failures. Empirical validation shows that our algorithm uses the failures budget more efficiently in a variety of optimization experiments, and generally achieves lower regret, than state-of-the-art methods. In addition, we propose an original algorithm for unconstrained Bayesian optimization inspired by the notion of excursion sets in stochastic processes, upon which the failures-aware algorithm is built.
arXiv code (python) PDF BibTeX
Proceedings Robotics: Science and System XVI RSS Foundation, Corvalis, OR, 2020 BibTeX
Proceedings An automated and self-initiated judgement bias task based on natural investigative behaviour Mendl, M., Jones, S., Neville, V., Higgs, L., Robinson, E., Dayan, P., Paul, E. Applied Ethology 2019: Animal lives worth living: 53rd Congress of the International Society of Applied Ethology (ISAE 2019), 126, Wageningen Academic Publishers, Wageningen, The Netherlands, 2019
{Scientific assessment of affective valence (positivity or negativity) in animals allows us to evaluate animal welfare and the effectiveness of 3Rs Refinements designed to improve wellbeing. Judgement bias tasks measure valence; however, task-training may be lengthy and/or require significant input from researchers. Here we develop an automated and self-initiated judgement bias task for rats which capitalises on their natural investigative behaviour. Rats insert their noses into a food trough recess to start trials. They then hear a tone (2 or 8 kHz) and learn either to \textquoteleftstay\textquoteright for 2 s to receive a food reward or to \textquoteleftleave\textquoteright the trough recess promptly to avoid an air-puff. Which contingency applies is signalled by two different tones. Judgement bias is measured by responses to intermediate ambiguous tones. We carried out two experiments to investigate this new task. In Experiment 1, 36 of 40 (90\textpercent) rats reached training criterion on the tone-discrimination task in a mean of 23.1 (sem: 1.14) sessions. Half the rats were partially-reinforced during training and they were more likely to \textquoteleftstay\textquoteright during ambiguous and negative tones than rats that were fully-reinforced (Likelihood-ratio test (LRT) of effect of removing predictor variable from model: Chi-square\textequals17.71, df\textequals1, P\textequals0.001). When exposed to prior short-term positive affect manipulations (15 min gentle handling; enrichment), rats tended to show more \textquoteleftstay\textquoteright responses (LRT Chi-square\textequals3.28, df\textequals1, P\textequals0.07) than when they were exposed to negative ones (15 min small box; isolation). In Experiment 2, all rats were partially-reinforced during training, and 11 of 12 (92\textpercent) rats reached criterion in 17.5 (sem: 0.65) sessions. Rats exposed to a prior short-term positive affect manipulation (16 food rewards in 15 min) tended to make more \textquoteleftstay\textquoteright responses (LRT Chi-square\textequals3.75, df\textequals1, P\textequals0.053) than those exposed to a relatively negative one (1 food reward in 15 min). This task capitalises on natural investigative behaviour, can be learnt in fewer sessions than other automated variants, generates 4-5 self-initiated trials/min, yields generalised responses across ambiguous tones as expected, and can be tested repeatedly. Affect manipulations generate main effect trends in the predicted directions, albeit not quite significant at P\textless0.05, and not localised to ambiguous tones perhaps indicating that affect manipulations altered food and/or air-puff valuation which influenced responses to all tones. Further construct validation is thus required. We also find that reinforcement contingencies during training can affect responses to ambiguity. The task is likely to be readily translatable to other species and should facilitate more widespread uptake of judgement bias testing.}
DOI BibTeX
Proceedings Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning Ahilan, S., Dayan, P. 4th Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM 2019), 57, 2019
{We investigate how reinforcement learning agents can learn to cooperate. Drawing inspiration from human societies, in which successful coordination of many individuals is often facilitated by hierarchical organisation, we introduce Feudal Multi-agent Hierarchies (FMH). In this framework, a \textquoteleftmanager\textquoteright agent, which is tasked with maximising the environmentally-determined reward function, learns to communicate subgoals to multiple, simultaneously-operating, \textquoteleftworker\textquoteright agents. Workers, which are rewarded for achieving managerial subgoals, take concurrent actions in the world. We outline the structure of FMH and demonstrate its potential for decentralised learning and control. We find that, given an adequate set of subgoals from which to choose, FMH performs, and particularly scales, substantially better than cooperative approaches that use a shared reward function.}
BibTeX
Proceedings Investigating animal affect and welfare using computational modelling Neville, V., Paul, L., Dayan, P., Gilchrist, I., Mendl, M. Applied Ethology 2019: Animal lives worth living: 53rd Congress of the International Society of Applied Ethology (ISAE 2019), 127, Wageningen Academic Publishers, Wageningen, The Netherlands, 2019
{Behaviour associated with poor welfare, such as \textquoteleftpessimistic\textquoteright decision-making, can arise from several different affect-induced shifts in cognitive function. For example, risk aversion can arise from an altered sensitivity to, or expectation of, rewards or punishers, and these processes can themselves be influenced by several environmental factors. By characterising the cognitive processes that generate behaviour, we can gain a better insight into the relationship between specific forms of adversity and indicators of welfare such as judgement bias. We aimed to use computational modelling to extract parameters relating to different aspects of cognitive processing from judgement bias decision-making data and to assess how these were influenced by reward experience, following the prediction that enhanced reward experience generates a positive affective state. To achieve this, we used an automated and self-initiated judgement bias task in which rats had to choose between a risky option which resulted in either an airpuff or apple juice, and a safe option which provided nothing. More specifically, rats initiated each trial by putting their nose in a trough which resulted in the immediate presentation of a tone, the frequency of which provided clear or ambiguous information about the potential outcome. Rats then either stayed in the trough for 2 s (\textquoteleftstay\textquoteright\textequalsrisky option) or removed their nose (\textquoteleftleave\textquoteright\textequalssafe option). We manipulated reward experience by systematically varying the volume of juice in a sinusoidal manner (mean\textequals1 ml, SD\textequals0.3 ml). Rats were not water or food restricted as part of these studies And all rats were rehomed as pets at the end of the study. These experiments adhered to the ISAE and ASAB/ABS guidelines for the ethical use of animals in research. Following data collection, we modelled decision-making on the task (binary variable: \textquoteleftstay\textquoteright or \textquoteleftleave\textquoteright) as a partially-observable Markov decision process with a two-dimensional state space describing each rat\textquoterights perception of the tone and time left to make a decision. The model provided a good fit of the data (RMSEA\textequals0.028). The computational analysis revealed that variation in risk aversion could be attributed to changes in prior beliefs about the likelihood of reward which was modulated by what an individual had learnt from previous outcomes in the test environment. Specifically, an individual\textquoterights expectation that the trial would be rewarded prior to presentation of the tone was greater when they had learnt that they were in a high reward environment, assumed to generate positive affect, resulting in more \textquoteleftoptimistic\textquoteright decision-making (dAIC\textequals4.979, P\textless0.001). As such, these models inform our understanding of the relationship between the environment, affect, and decision-making. The parameters obtained using this approach may provide a more precise measure of welfare than the decision itself and hence provide a better estimate of the affective impact of poor or improved husbandry. Computational modelling can be a useful tool in the study of animal welfare.}
DOI BibTeX
Proceedings Game room map integration in virtual environments for free walking Keller, M., Exposito, F. 763-764, 2018
Current tracking systems now enable real walking in a virtual scene with a Head Mounted Display (HMD). However, the play area usually remains limited to a few square meters because of tracking limits and the lack of free space in game rooms. This paper describes our demonstration showing how we can use an RGB-D sensor to increase the real game surface by dynamically acquiring a map of the actual game room and integrating it into the virtual environment. Our system was designed to be integrable to any virtual environment and aims to enable free walking with a HMD by showing the position of the real obstacles to the user.
Paper DOI BibTeX
Proceedings 3rd Workshop on Eye Tracking and Visualization (ETVIS \textquotesingle18) 53, Warsaw, Poland, 2018 BibTeX
Proceedings KOGWIS2018: Computational Approaches to Cognitive Science Technische Universität Darmstadt, Darmstadt, Germany, 2018 BibTeX
Empirical Inference Proceedings Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (UAI) Ihler, A. T., Janzing, D. 869 pages, AUAI Press, June 2016 (Published) URL BibTeX
Perceiving Systems Proceedings Proceedings of the 37th German Conference on Pattern Recognition Gall, J., Gehler, P., Leibe, B. Springer, German Conference on Pattern Recognition, October 2015 GCPR conference website BibTeX
Empirical Inference Proceedings Proceedings of the 10th European Workshop on Reinforcement Learning, Volume 24 Deisenroth, M., Szepesvári, C., Peters, J. 173, JMLR, European Workshop On Reinforcement Learning, EWRL, 2013 Web BibTeX
Autonomous Motion Proceedings 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), Osaka, Japan, November 29 - Dec. 1, 2012 IEEE, 2012 URL BibTeX
Empirical Inference Proceedings MICCAI, Workshop on Computational Diffusion MRI, 2012 (electronic publication) Panagiotaki, E., O’Donnell, L., Schultz, T., Zhang, G. 15th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2012), Workshop on Computational Diffusion MRI , 2012 PDF BibTeX
Empirical Inference Proceedings Machine Learning and Interpretation in Neuroimaging - Revised Selected and Invited Contributions Langs, G., Rish, I., Grosse-Wentrup, M., Murphy, B. 266, Springer, Heidelberg, Germany, International Workshop, MLINI 2011, Held at NIPS, 2012, Lecture Notes in Computer Science, Vol. 7263 DOI BibTeX
Empirical Inference Proceedings JMLR Workshop and Conference Proceedings Volume 19: COLT 2011 Kakade, S., von Luxburg, U. 834, MIT Press, Cambridge, MA, USA, 24th Annual Conference on Learning Theory , June 2011 Web BibTeX
Autonomous Motion Proceedings 11th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2011), Bled, Slovenia, October 26-28, 2011 IEEE, 2011 URL BibTeX
Empirical Inference Proceedings JMLR Workshop and Conference Proceedings: Volume 6 Guyon, I., Janzing, D., Schölkopf, B. 288, MIT Press, Cambridge, MA, USA, Causality: Objectives and Assessment (NIPS 2008 Workshop) , February 2010 Web BibTeX
Empirical Inference Proceedings CogRob 2008: The 6th International Cognitive Robotics Workshop Lespérance, Y., Lakemeyer, G., Peters, J., Pirri, F. Proceedings of the 6th International Cognitive Robotics Workshop (CogRob 2008), 35, Patras University Press, Patras, Greece, 6th International Cognitive Robotics Workshop (CogRob 2008), July 2008 Web BibTeX
Empirical Inference Proceedings Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference Schölkopf, B., Platt, J., Hofmann, T. Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems (NIPS 2006), 1690, MIT Press, Cambridge, MA, USA, 20th Annual Conference on Neural Information Processing Systems (NIPS 2006), September 2007
The annual Neural Information Processing Systems (NIPS) conference is the flagship meeting on neural computation and machine learning. It draws a diverse group of attendees--physicists, neuroscientists, mathematicians, statisticians, and computer scientists--interested in theoretical and applied aspects of modeling, simulating, and building neural-like or intelligent systems. The presentations are interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, brain imaging, vision, speech and signal processing, reinforcement learning, and applications. Only twenty-five percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. This volume contains the papers presented at the December 2006 meeting, held in Vancouver.
Web BibTeX
Empirical Inference Proceedings Advances in Neural Information Processing Systems 18: Proceedings of the 2005 Conference Weiss, Y., Schölkopf, B., Platt, J. Proceedings of the 19th Annual Conference on Neural Information Processing Systems (NIPS 2005), 1676, MIT Press, Cambridge, MA, USA, 19th Annual Conference on Neural Information Processing Systems (NIPS 2005), May 2006
The annual Neural Information Processing Systems (NIPS) conference is the flagship meeting on neural computation. It draws a diverse group of attendees--physicists, neuroscientists, mathematicians, statisticians, and computer scientists. The presentations are interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, brain imaging, vision, speech and signal processing, reinforcement learning and control, emerging technologies, and applications. Only twenty-five percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. This volume contains the papers presented at the December 2005 meeting, held in Vancouver.
Web BibTeX
Empirical Inference Proceedings Machine Learning Challenges: evaluating predictive uncertainty, visual object classification and recognising textual entailment Quinonero Candela, J., Dagan, I., Magnini, B., Lauria, F. Proceedings of the First Pascal Machine Learning Challenges Workshop on Machine Learning Challenges, Evaluating Predictive Uncertainty, Visual Object Classification and Recognizing Textual Entailment (MLCW 2005), 462, Lecture Notes in Computer Science, Springer, Heidelberg, Germany, First Pascal Machine Learning Challenges Workshop (MLCW 2005), 2006
This book constitutes the thoroughly refereed post-proceedings of the First PASCAL (pattern analysis, statistical modelling and computational learning) Machine Learning Challenges Workshop, MLCW 2005, held in Southampton, UK in April 2005. The 25 revised full papers presented were carefully selected during two rounds of reviewing and improvement from about 50 submissions. The papers reflect the concepts of three challenges dealt with in the workshop: finding an assessment base on the uncertainty of predictions using classical statistics, Bayesian inference, and statistical learning theory; the second challenge was to recognize objects from a number of visual object classes in realistic scenes; the third challenge of recognizing textual entailment addresses semantic analysis of language to form a generic framework for applied semantic inference in text understanding.
Web DOI BibTeX
Empirical Inference Proceedings Advanced Lectures on Machine Learning Bousquet, O., von Luxburg, U., Rätsch, G. ML Summer Schools 2003, LNAI 3176:240, Springer, Berlin, Germany, ML Summer Schools, September 2004
Machine Learning has become a key enabling technology for many engineering applications, investigating scientific questions and theoretical problems alike. To stimulate discussions and to disseminate new results, a summer school series was started in February 2002, the documentation of which is published as LNAI 2600. This book presents revised lectures of two subsequent summer schools held in 2003 in Canberra, Australia, and in T{\"u}bingen, Germany. The tutorial lectures included are devoted to statistical learning theory, unsupervised learning, Bayesian inference, and applications in pattern recognition; they provide in-depth overviews of exciting new developments and contain a large number of references. Graduate students, lecturers, researchers and professionals alike will find this book a useful resource in learning and teaching machine learning.
Web BibTeX
Empirical Inference Proceedings Pattern Recognition: 26th DAGM Symposium, LNCS, Vol. 3175 Rasmussen, C., Bülthoff, H., Giese, M., Schölkopf, B. Proceedings of the 26th Pattern Recognition Symposium (DAGM‘04), 581, Springer, Berlin, Germany, 26th Pattern Recognition Symposium, August 2004 Web DOI BibTeX
Empirical Inference Proceedings Advances in Neural Information Processing Systems 16: Proceedings of the 2003 Conference Thrun, S., Saul, L., Schölkopf, B. Proceedings of the Seventeenth Annual Conference on Neural Information Processing Systems (NIPS 2003), 1621, MIT Press, Cambridge, MA, USA, 17th Annual Conference on Neural Information Processing Systems (NIPS 2003), June 2004
The annual Neural Information Processing (NIPS) conference is the flagship meeting on neural computation. It draws a diverse group of attendees—physicists, neuroscientists, mathematicians, statisticians, and computer scientists. The presentations are interdisciplinary, with contributions in algorithms, learning theory, cognitive science, neuroscience, brain imaging, vision, speech and signal processing, reinforcement learning and control, emerging technologies, and applications. Only thirty percent of the papers submitted are accepted for presentation at NIPS, so the quality is exceptionally high. This volume contains all the papers presented at the 2003 conference.
Web BibTeX
Empirical Inference Proceedings Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop (COLT/Kernel 2003), LNCS Vol. 2777 Schölkopf, B., Warmuth, M. Proceedings of the 16th Annual Conference on Learning Theory and 7th Kernel Workshop (COLT/Kernel 2003), COLT/Kernel 2003, 746, Springer, Berlin, Germany, 16th Annual Conference on Learning Theory and 7th Kernel Workshop, November 2003, Lecture Notes in Computer Science ; 2777 DOI BibTeX