Aamir Ahmad

Research Scientist

Perceiving Systems

Office: N3.007
Max-Planck-Ring 4
72076 Tübingen
Germany

+49 7071 601 1816

I am leading the Robot Perception Group at the Perceiving Systems Department. We have developed MPC-based formation-control methods and algorithms for multiple micro aerial vehicles (MAVs) that enable deep methods for person detection run in realtime on light-weight on-board GPUs on every MAV, to jointly perceive moving persons. More recently we have developed a convex MPC-based active perception-based formation control method. This method allows our MAVs to arrive at formations that jointly minimize the uncertainy in target perception while respecting other formation constraints, especially that of collision avoidance. For further information, please visit my group page and other project pages.

Please find the latest version of my CV here.

For videos of my research please visit my youtube channel here

News

[Dataset and Code of our ICCV 2019 paper] 'Markerless Outdoor Human Motion Capture Using Multiple Autonomous Micro Aerial Vehicles' -- Dataset and code here.
[IEEE/RSJ IROS 2019] Our talk in IROS Workshop on Aerial Swarms is now online!
[IEEE/RSJ IROS 2019] Our submission to IROS Workshop on Aerial Swarms is accepted! To know all about the AirCap project please attend our talk on Nov 4.
[Call for a PhD Position (Closed) - Deadline: ~~1st September~~] We have a new PhD student position open on the topic 'Reinforcement Learning for Aerial Robots'. See the call here, or contact me directly.
[Paper accepted in ICCV 2019] Our latest work titled 'Markerless Outdoor Human Motion Capture Using Multiple Autonomous Micro Aerial Vehicles' has been accepted to ICCV (July, 2019)
[Paper accepted in IEEE RA-L 2019] Our latest work titled 'Active Perception based Formation Control for Multiple Aerial Vehicles' has been accepted as a journal paper in IEEE Robotics and Automation Letters (July, 2019)
[Paper accepted in IEEE CASE 2019] Rahul's work titled 'Motion Planning for Multi-Mobile-Manipulator Payload Transport Systems' has been accepted as a conference paper in IEEE 15th International Conference on Automation Science and Engineering (CASE) (May, 2019)
[EU H2020 Grant proposal DeepField accepted for funding] Our grant proposal, from a 5-member consortium consisting of several European Universities, is accepted for funding. Project begins October 2019. Stay tuned for more info.
[Code Release] : Nodes and packages specific to our RA-L paper (accepted) "Active Target Perception based Formation Control for Multiple Micro Aerial Vehicles" is added on our Github project page. AirCap Github Page
[Talk] NVidia I AM AI Photoseries featured my short interview with them at the GTC 2018.
[Paper accepted in IEEE/RSJ IROS 2018] Our RA-L + IROS 2018 submission from the AirCap project got accepted as IROS paper also. Please see the publications for RA-L version of the paper.
[Paper accepted in IEEE SSRR 2018] Our work on decentralized MPC with integrated obstacle avoidance for multiple UAVs got accepted at SSRR 2018.
[Paper accepted in IEEE RA-L 2018] Our latest work and results from the AirCap project got accepted as a journal paper in IEEE Robotics and Automation Letters (RA-L). Please check my publications tab.
My talk regarding outdoor motion capture using UAVs at 2018 GPU Technology Conference (GTC), March 2018, San José, USA, now online here.
I was a featured speaker in the autonomous machines session at the 2018 GPU Technology Conference (GTC).

Research Overview

Please see my group page here -- Robot Perception Group for my research overview.

New Robot Platforms -- In order to have extensive access to the hardware, we design and build most of our robotic platforms. Currently, our main flying platforms include 8-rotor Octocopters. More details can be found here https://ps.is.tue.mpg.de/pages/outdoor-aerial-motion-capture-system.

There are currently two ongoing AirCap projects in our group: 3D Motion Capture and Perception-Based Control

PhD Students

Eric Price (Co-supervised, Main Supervisor: Michael Black; September 2016 -- present)
Nitin Saini (Co-supervised, Main Supervisor: Michael Black; April 2018 -- present)
Rahul Tallamraju (Co-supervised, Main Supervisor: Professor Kamalakar Karlapalem)

PhD Interns

Rahul Tallamraju (March 2018 -- present)
Guilherme Lawless (up to Sep' 2017, currently collaborating from ISR/IST, Lisbon)

Master/Bachelor Students

Igor Martinović (September 2017 -- present)
Halil Acet (July 2019 -- present)
Nowfal Manakkaparambil Ali (July 2019 -- present)
Yu-Tang Liu (August 2019 -- present)
Ivy Nuo Chen (August 2019 -- present)

Previous supervisions

Roman Ludwig (September 2017 -- July 2019), (Research Assistant, Currently a PhD student at ETH Zürich)
Eugen Ruff (Research Engineer, Currently at Bosch)
Soumyadeep Mukherjee (Bachelor Intern)
Raman Butta (Bachelor Intern)
David Sanz (PhD Intern)

Invited Talks at International Conferences

Deep Neural Network-based Cooperative Visual Tracking through Multiple Flying Robots, 2018 GPU Technology Conference (GTC), March 2018, San José, USA.
- Listen to the talk and see slides here.
Human-Multirobot Interaction in Cooperative Perception-based Search and Rescue Missions, 2017 IEEE ICRA Workshop on Human Multi-Robot Systems Interaction, June 2, 2017, Singapore.
- Find the talk's schedule here.

Invited Lectures at Summer Schools

Cooperative Perception, "Lucia" PhD School on AI and Robotics, Sep 4-8, 2017, Lisbon, Portugal.
- Find school's program here.
- Youtube link of the talk.
Cooperative Robot Localization and Target Tracking based on Least Squares Minimization, LARSyS Summer School on Multi-agent Systems, July 8-11, 2014, Lisbon, Portugal.

Previous Projects

TRaVERSE: TowaRds Very large scalE human-Robot SynErgy- (Funded by EU-FP7 Marie Curie IEF) -- Hazardous work environment for humans, growing necessity for an increase in the worldwide agricultural production, and a rapid rise in the public healthcare expenditures due to aging population are among some of the most predominant societal issues where robotics and automation are becoming progressively vital. Distributed multi-robot teams consisting of a large number of robots operating in close cooperation with humans is fundamental for such applications. How to achieve maximum synergy between teams of robots and humans, without jeopardizing human safety and comfort, within the constraints of resources, e.g., computational capacity of the robots, sensor and actuator costs, is still an open question. The focus of this research project is on investigating and developing integrated methods for robot team functionalities with human interaction that are scalable to a very large number of robots, thus enabling their successful real-world deployment. Using state of the art, ecologically valid and immersive virtual environments and virtual reality equipments, the project will study and model human behavior, perception and cognitive responses when they interact and/or cooperate with robots in a large-scale multi-robot scenario. Clear distinction will be made between human users and operators where the former is expected to benefit from directly using a robot that functions as a part of a large robotic team. On the other hand, human operators are those that are in charge of cooperating/controlling a team of robots to accomplish a collaborative task. Eventually, using the human behavior models, cooperative multi-robot functionalities, including localization, mapping and motion planning, will be optimally designed to be extremely scalable within the constraints of ease, safety, effectiveness and naturalness of human user/operator interaction with the robots.
RoCKIn: Robot Competitions Kick Innovation in Cognitive Systems and Robotics (FP7-EU-601012) [Quoted from the project's website] “RoCKIn is an EU project that will be run over the next three years (2013-2016), consisting of robot competitions, symposiums, educational RoCKIn camps and technology transfer workshops. The mission is to act as a catalyst for smarter, more dependable robots. It is done by building upon the principles of challenge-driven innovation laid down by RoboCup, facilitating cognitive and networked robot systems' testing, and streamlining research and development through standardised testbeds and benchmarks. For this there are two challenges which will run concurrently in 2014 and 2015 with an introductory event to be held in June 2013. These challenges were selected due to their high relevance and impact on Europe's societal and industrial needs.” http://rockinrobotchallenge.eu/
PCMMC: Perception-Driven Coordinated Multi-Robot Motion Control (FCT PTDC/EEA-CRO/100692/2008) [Quoted from the project's wiki] Several robotic tasks require or benefit from the cooperation of multiple robots: transportation of large-size objects, large area coverage (e.g., for cleaning) or surveillance (e.g., for fire detection), pollutant plume tracking, or target detection and tracking, to name but a few. In this project, a novel active approach to cooperative perception through coordinated vehicle motion control is proposed. The vehicle formation geometry will change dynamically so as to optimize the accuracy of cooperative perception of a static or dynamic target by the formation vehicles. To achieve this, innovative decentralized low-communication formation full state estimation methods, and dynamic-goal-driven formation control, for cooperative target localization and tracking by decentralized fusion of the data measured by all the formation vehicles have been introduced. http://mediawiki.isr.ist.utl.pt/wiki/PCMMC:_Perception-Driven_Coordinated_Multi-Robot_Motion_Control
SocRob [Quoted from the project's website] SocRob is a project on Cooperative Robotics and Multi-Agent Systems carried out by the Intelligent Systems Laboratory at ISR/IST. The acronym of the project stands both for “Society of Robots” and “Soccer Robots”, the case study where we are testing our population of four robots. http://socrob.isr.ist.utl.pt/
URUS: Ubiquitous Networking Robotics in Urban Settings [Quoted from the project's website] In this project the idea of incorporating a network of robots (robots, intelligent sensors, devices and communications) in order to improve life quality in urban areas was analyzed and tested. The URUS project was focused in designing a network of robots that in a cooperative way interact with human beings and the environment for tasks of assistance, transportation of goods, and surveillance in urban areas. Specifically, the objective was to design and develop a cognitive network robot architecture that integrates cooperating urban robots, intelligent sensors, intelligent devices and communications. http://urus.upc.es/

A big chunk of code and dataset that I contribute to is open source and can be accessed through my github page mentioned on the left-side of this page. Some of my favorite ones are described below.

Datasets:

Omni-dataset: Time-stamped sensor data acquired from four omni-directional soccer robots of the SocRob team can be downloaded from here. The dataset also contains groundtruth information acquired via an external vision system.
DSR dataset: Domestic service robot dataset. It consists of time-stamped sensor data acquired by one service robot moving around in a real home-like environment while a person randomly walks around in the same environment. The dataset can be downloaded from here. Ground truth information is also provided.

Software:

ROS package 'read_omni_dataset' to systematically read sensor messages from the omni-dataset: Download the source here.
ROS package 'evaluate_omni_dataset' to do GT evaluation and benchamarking with the omni-dataset and is expected to be executed with 'read_omni_dataset'. Download the source here.
ROS package 'read_dsr_dataset' to systematically read sensor messages from the DSR dataset: Download the source here.
ROS package 'evaluate_dsr_dataset' to do GT evaluation and benchamarking with the DSR dataset and is expected to be executed with 'read_dsr_dataset'. Download the source here.

AirCap Story

The story of our project AirCap: Aerial Outdoor Motion Capture. The future of motion capture is here!

AirCap: 3D Motion Capture

Our goal is markless, unconstrained, human and animal motion capture outdoors. To that end, we are developing a flying mocap system using a team of aerial vehicles (MAVs) with only on-board, monocular RGB cameras. To realize such an outdoor motion capture system we need to address research challenges...

Aamir Ahmad Eric Price Nitin Saini Guilherme Lawless Roman Ludwig Igor Martinovic Michael Black Aamir Ahmad

AirCap: Perception-Based Control

Autonomous MoCap systems, like AirCap, rely on robots with on-board cameras that can localize and navigate autonomously. More importantly, these robots must detect, track and follow the subject (human or animal) in real time. Thus, a key component of such a system is motion planning and control of multiple...

Aamir Ahmad Eric Price Rahul Tallamraju Roman Ludwig Nowfal Manakkaparambil Ali Igor Martinovic Yu-Tang Liu Ivy Nuo Chen Michael Pabst Halil Acet Michael Black Aamir Ahmad

AirCap: 3D Motion Capture

Aamir Ahmad Eric Price Nitin Saini Guilherme Lawless Roman Ludwig Igor Martinovic Michael Black Aamir Ahmad

AirCap: Perception-Based Control

Aamir Ahmad Eric Price Rahul Tallamraju Roman Ludwig Nowfal Manakkaparambil Ali Igor Martinovic Yu-Tang Liu Ivy Nuo Chen Michael Pabst Halil Acet Michael Black Aamir Ahmad

34 results (View BibTeX file of all listed publications)

2019

AirCap – Aerial Outdoor Motion Capture

Ahmad, A., Price, E., Tallamraju, R., Saini, N., Lawless, G., Ludwig, R., Martinovic, I., Bülthoff, H. H., Black, M. J.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019), Workshop on Aerial Swarms, November 2019 (misc)

Abstract

This paper presents an overview of the Grassroots project Aerial Outdoor Motion Capture (AirCap) running at the Max Planck Institute for Intelligent Systems. AirCap's goal is to achieve markerless, unconstrained, human motion capture (mocap) in unknown and unstructured outdoor environments. To that end, we have developed an autonomous flying motion capture system using a team of aerial vehicles (MAVs) with only on-board, monocular RGB cameras. We have conducted several real robot experiments involving up to 3 aerial vehicles autonomously tracking and following a person in several challenging scenarios using our approach of active cooperative perception developed in AirCap. Using the images captured by these robots during the experiments, we have demonstrated a successful offline body pose and shape estimation with sufficiently high accuracy. Overall, we have demonstrated the first fully autonomous flying motion capture system involving multiple robots for outdoor scenarios.

Talk slides Project Page Project Page [BibTex]

2019

ps Ahmad, A., Price, E., Tallamraju, R., Saini, N., Lawless, G., Ludwig, R., Martinovic, I., Bülthoff, H. H., Black, M. J. AirCap – Aerial Outdoor Motion Capture IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019), Workshop on Aerial Swarms, November 2019 (misc)

Talk slides Project Page Project Page [BibTex]

Markerless Outdoor Human Motion Capture Using Multiple Autonomous Micro Aerial Vehicles

Saini, N., Price, E., Tallamraju, R., Enficiaud, R., Ludwig, R., Martinović, I., Ahmad, A., Black, M.

In International Conference on Computer Vision, October 2019 (inproceedings) Accepted

Abstract

Capturing human motion in natural scenarios means moving motion capture out of the lab and into the wild. Typical approaches rely on fixed, calibrated, cameras and reflective markers on the body, significantly limiting the motions that can be captured. To make motion capture truly unconstrained, we describe the first fully autonomous outdoor capture system based on flying vehicles. We use multiple micro-aerial-vehicles(MAVs), each equipped with a monocular RGB camera, an IMU, and a GPS receiver module. These detect the person, optimize their position, and localize themselves approximately. We then develop a markerless motion capture method that is suitable for this challenging scenario with a distant subject, viewed from above, with approximately calibrated and moving cameras. We combine multiple state-of-the-art 2D joint detectors with a 3D human body model and a powerful prior on human pose. We jointly optimize for 3D body pose and camera pose to robustly fit the 2D measurements. To our knowledge, this is the first successful demonstration of outdoor, full-body, markerless motion capture from autonomous flying vehicles.

Code Data Video Paper Manuscript Project Page [BibTex]

ps Saini, N., Price, E., Tallamraju, R., Enficiaud, R., Ludwig, R., Martinović, I., Ahmad, A., Black, M. Markerless Outdoor Human Motion Capture Using Multiple Autonomous Micro Aerial Vehicles In International Conference on Computer Vision, October 2019 (inproceedings) Accepted

Code Data Video Paper Manuscript Project Page [BibTex]

Active Perception based Formation Control for Multiple Aerial Vehicles

Tallamraju, R., Price, E., Ludwig, R., Karlapalem, K., Bülthoff, H. H., Black, M. J., Ahmad, A.

IEEE Robotics and Automation Letters, Robotics and Automation Letters, 4(4):4491-4498, IEEE, October 2019 (article)

Abstract

We present a novel robotic front-end for autonomous aerial motion-capture (mocap) in outdoor environments. In previous work, we presented an approach for cooperative detection and tracking (CDT) of a subject using multiple micro-aerial vehicles (MAVs). However, it did not ensure optimal view-point configurations of the MAVs to minimize the uncertainty in the person's cooperatively tracked 3D position estimate. In this article, we introduce an active approach for CDT. In contrast to cooperatively tracking only the 3D positions of the person, the MAVs can actively compute optimal local motion plans, resulting in optimal view-point configurations, which minimize the uncertainty in the tracked estimate. We achieve this by decoupling the goal of active tracking into a quadratic objective and non-convex constraints corresponding to angular configurations of the MAVs w.r.t. the person. We derive this decoupling using Gaussian observation model assumptions within the CDT algorithm. We preserve convexity in optimization by embedding all the non-convex constraints, including those for dynamic obstacle avoidance, as external control inputs in the MPC dynamics. Multiple real robot experiments and comparisons involving 3 MAVs in several challenging scenarios are presented.

pdf DOI Project Page [BibTex]

ps Tallamraju, R., Price, E., Ludwig, R., Karlapalem, K., Bülthoff, H. H., Black, M. J., Ahmad, A. Active Perception based Formation Control for Multiple Aerial Vehicles IEEE Robotics and Automation Letters, Robotics and Automation Letters, 4(4):4491-4498, IEEE, October 2019 (article)

pdf DOI Project Page [BibTex]

Motion Planning for Multi-Mobile-Manipulator Payload Transport Systems

Tallamraju, R., Salunkhe, D., Rajappa, S., Ahmad, A., Karlapalem, K., Shah, S. V.

In 15th IEEE International Conference on Automation Science and Engineering, IEEE, August 2019 (inproceedings) Accepted

[BibTex]

ps Tallamraju, R., Salunkhe, D., Rajappa, S., Ahmad, A., Karlapalem, K., Shah, S. V. Motion Planning for Multi-Mobile-Manipulator Payload Transport Systems In 15th IEEE International Conference on Automation Science and Engineering, IEEE, August 2019 (inproceedings) Accepted

[BibTex]

2018

Deep Neural Network-based Cooperative Visual Tracking through Multiple Micro Aerial Vehicles

Price, E., Lawless, G., Ludwig, R., Martinovic, I., Buelthoff, H. H., Black, M. J., Ahmad, A.

IEEE Robotics and Automation Letters, Robotics and Automation Letters, 3(4):3193-3200, IEEE, October 2018, Also accepted and presented in the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). (article)

Abstract

Multi-camera tracking of humans and animals in outdoor environments is a relevant and challenging problem. Our approach to it involves a team of cooperating micro aerial vehicles (MAVs) with on-board cameras only. DNNs often fail at objects with small scale or far away from the camera, which are typical characteristics of a scenario with aerial robots. Thus, the core problem addressed in this paper is how to achieve on-board, online, continuous and accurate vision-based detections using DNNs for visual person tracking through MAVs. Our solution leverages cooperation among multiple MAVs and active selection of most informative regions of image. We demonstrate the efficiency of our approach through simulations with up to 16 robots and real robot experiments involving two aerial robots tracking a person, while maintaining an active perception-driven formation. ROS-based source code is provided for the benefit of the community.

Published Version link (url) DOI [BibTex]

2018

ps Price, E., Lawless, G., Ludwig, R., Martinovic, I., Buelthoff, H. H., Black, M. J., Ahmad, A. Deep Neural Network-based Cooperative Visual Tracking through Multiple Micro Aerial Vehicles IEEE Robotics and Automation Letters, Robotics and Automation Letters, 3(4):3193-3200, IEEE, October 2018, Also accepted and presented in the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). (article)

Published Version link (url) DOI [BibTex]

Decentralized MPC based Obstacle Avoidance for Multi-Robot Target Tracking Scenarios

Tallamraju, R., Rajappa, S., Black, M. J., Karlapalem, K., Ahmad, A.

2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pages: 1-8, IEEE, August 2018 (conference)

Abstract

In this work, we consider the problem of decentralized multi-robot target tracking and obstacle avoidance in dynamic environments. Each robot executes a local motion planning algorithm which is based on model predictive control (MPC). The planner is designed as a quadratic program, subject to constraints on robot dynamics and obstacle avoidance. Repulsive potential field functions are employed to avoid obstacles. The novelty of our approach lies in embedding these non-linear potential field functions as constraints within a convex optimization framework. Our method convexifies nonconvex constraints and dependencies, by replacing them as pre-computed external input forces in robot dynamics. The proposed algorithm additionally incorporates different methods to avoid field local minima problems associated with using potential field functions in planning. The motion planner does not enforce predefined trajectories or any formation geometry on the robots and is a comprehensive solution for cooperative obstacle avoidance in the context of multi-robot target tracking. We perform simulation studies for different scenarios to showcase the convergence and efficacy of the proposed algorithm.

Published Version link (url) DOI [BibTex]

ps Tallamraju, R., Rajappa, S., Black, M. J., Karlapalem, K., Ahmad, A. Decentralized MPC based Obstacle Avoidance for Multi-Robot Target Tracking Scenarios 2018 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pages: 1-8, IEEE, August 2018 (conference)

Published Version link (url) DOI [BibTex]

2017

An Online Scalable Approach to Unified Multirobot Cooperative Localization and Object Tracking

Ahmad, A., Lawless, G., Lima, P.

IEEE Transactions on Robotics (T-RO), 33, pages: 1184 - 1199, October 2017 (article)

Abstract

In this article we present a unified approach for multi-robot cooperative simultaneous localization and object tracking based on particle filters. Our approach is scalable with respect to the number of robots in the team. We introduce a method that reduces, from an exponential to a linear growth, the space and computation time requirements with respect to the number of robots in order to maintain a given level of accuracy in the full state estimation. Our method requires no increase in the number of particles with respect to the number of robots. However, in our method each particle represents a full state hypothesis, leading to the linear dependency on the number of robots of both space and time complexity. The derivation of the algorithm implementing our approach from a standard particle filter algorithm and its complexity analysis are presented. Through an extensive set of simulation experiments on a large number of randomized datasets, we demonstrate the correctness and efficacy of our approach. Through real robot experiments on a standardized open dataset of a team of four soccer playing robots tracking a ball, we evaluate our method's estimation accuracy with respect to the ground truth values. Through comparisons with other methods based on i) nonlinear least squares minimization and ii) joint extended Kalman filter, we further highlight our method's advantages. Finally, we also present a robustness test for our approach by evaluating it under scenarios of communication and vision failure in teammate robots.

Published Version link (url) DOI [BibTex]

2017

ps Ahmad, A., Lawless, G., Lima, P. An Online Scalable Approach to Unified Multirobot Cooperative Localization and Object Tracking IEEE Transactions on Robotics (T-RO), 33, pages: 1184 - 1199, October 2017 (article)

Published Version link (url) DOI [BibTex]

2016

Dynamic baseline stereo vision-based cooperative target tracking

Ahmad, A., Ruff, E., Bülthoff, H.

19th International Conference on Information Fusion, pages: 1728-1734, July 2016 (conference)

Abstract

In this article we present a new method for multi-robot cooperative target tracking based on dynamic baseline stereo vision. The core novelty of our approach includes a computationally light-weight scheme to compute the 3D stereo measurements that exactly satisfy the epipolar constraints and a covariance intersection (CI)-based method to fuse the 3D measurements obtained by each individual robot. Using CI we are able to systematically integrate the robot localization uncertainties as well as the uncertainties in the measurements generated by the monocular camera images from each individual robot into the resulting stereo measurements. Through an extensive set of simulation and real robot results we show the robustness and accuracy of our approach with respect to ground truth. The source code related to this article is publicly accessible on our website and the datasets are available on request.

Aamir Ahmad

News

Research Overview

PhD Students

PhD Interns

Master/Bachelor Students

Previous supervisions

Invited Talks at International Conferences

Invited Lectures at Summer Schools

Previous Projects

AirCap Story

2019

2019

2018

2018

2017

2017

2016

2016

2015

2015

2014

2014

2013

2013

2011

2011

2009

2009

2008

2008

Latest News

Links

Contact Us