Header logo is


2020


no image
Algorithmic Recourse: from Counterfactual Explanations to Interventions

Karimi, A., Schölkopf, B., Valera, I.

37th International Conference on Machine Learning (ICML), July 2020 (conference) Submitted

ei plg

[BibTex]

2020


[BibTex]


Learning of sub-optimal gait controllers for magnetic walking soft millirobots
Learning of sub-optimal gait controllers for magnetic walking soft millirobots

Culha, U., Demir, S. O., Trimpe, S., Sitti, M.

In Proceedings of Robotics: Science and Systems, July 2020, Culha and Demir are equally contributing authors (inproceedings)

Abstract
Untethered small-scale soft robots have promising applications in minimally invasive surgery, targeted drug delivery, and bioengineering applications as they can access confined spaces in the human body. However, due to highly nonlinear soft continuum deformation kinematics, inherent stochastic variability during fabrication at the small scale, and lack of accurate models, the conventional control methods cannot be easily applied. Adaptivity of robot control is additionally crucial for medical operations, as operation environments show large variability, and robot materials may degrade or change over time,which would have deteriorating effects on the robot motion and task performance. Therefore, we propose using a probabilistic learning approach for millimeter-scale magnetic walking soft robots using Bayesian optimization (BO) and Gaussian processes (GPs). Our approach provides a data-efficient learning scheme to find controller parameters while optimizing the stride length performance of the walking soft millirobot robot within a small number of physical experiments. We demonstrate adaptation to fabrication variabilities in three different robots and to walking surfaces with different roughness. We also show an improvement in the learning performance by transferring the learning results of one robot to the others as prior information.

pi

link (url) DOI [BibTex]

link (url) DOI [BibTex]


no image
Model-Agnostic Counterfactual Explanations for Consequential Decisions

Karimi, A., Barthe, G., Balle, B., Valera, I.

Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), June 2020 (conference) Accepted

ei plg

arXiv [BibTex]

arXiv [BibTex]


SIMULTANEOUS CALIBRATION METHOD FOR MAGNETIC LOCALIZATION AND ACTUATION SYSTEMS
SIMULTANEOUS CALIBRATION METHOD FOR MAGNETIC LOCALIZATION AND ACTUATION SYSTEMS

Sitti, M., Son, D., Dong, X.

June 2020, US Patent App. 16/696,605 (misc)

Abstract
The invention relates to a method of simultaneously calibrating magnetic actuation and sensing systems for a workspace, wherein the actuation system comprises a plurality of magnetic actuators and the sensing system comprises a plurality of magnetic sensors, wherein all the measured data is fed into a calibration model, wherein the calibration model is based on a sensor measurement model and a magnetic actuation model, and wherein a solution of the model parameters is found via a numerical solver order to calibrate both the actuation and sensing systems at the same time.

pi

[BibTex]


FootTile: a Rugged Foot Sensor for Force and Center of Pressure Sensing in Soft Terrain
FootTile: a Rugged Foot Sensor for Force and Center of Pressure Sensing in Soft Terrain

Felix Ruppert, , Badri-Spröwitz, A.

In Proceedings of the IEEE International Conference on Robotics and Automation, IEEE, International Conference on Robotics and Automation, May 2020 (inproceedings) Accepted

Abstract
In this paper, we present FootTile, a foot sensor for reaction force and center of pressure sensing in challenging terrain. We compare our sensor design to standard biomechanical devices, force plates and pressure plates. We show that FootTile can accurately estimate force and pressure distribution during legged locomotion. FootTile weighs 0.9g, has a sampling rate of 330 Hz, a footprint of 10×10 mm and can easily be adapted in sensor range to the required load case. In three experiments, we validate: first, the performance of the individual sensor, second an array of FootTiles for center of pressure sensing and third the ground reaction force estimation during locomotion in granular substrate. We then go on to show the accurate sensing capabilities of the waterproof sensor in liquid mud, as a showcase for real world rough terrain use.

dlg

Youtube1 Youtube2 Presentation link (url) [BibTex]

Youtube1 Youtube2 Presentation link (url) [BibTex]


Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image
Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image

Paschalidou, D., Gool, L., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

Abstract
Humans perceive the 3D world as a set of distinct objects that are characterized by various low-level (geometry, reflectance) and high-level (connectivity, adjacency, symmetry) properties. Recent methods based on convolutional neural networks (CNNs) demonstrated impressive progress in 3D reconstruction, even when using a single 2D image as input. However, the majority of these methods focuses on recovering the local 3D geometry of an object without considering its part-based decomposition or relations between parts. We address this challenging problem by proposing a novel formulation that allows to jointly recover the geometry of a 3D object as a set of primitives as well as their latent hierarchical structure without part-level supervision. Our model recovers the higher level structural decomposition of various objects in the form of a binary tree of primitives, where simple parts are represented with fewer primitives and more complex parts are modeled with more components. Our experiments on the ShapeNet and D-FAUST datasets demonstrate that considering the organization of parts indeed facilitates reasoning about 3D geometry.

avg

pdf suppmat Video 2 Project Page Slides Poster Video 1 [BibTex]

pdf suppmat Video 2 Project Page Slides Poster Video 1 [BibTex]


Towards 5-DoF Control of an Untethered Magnetic Millirobot via MRI Gradient Coils
Towards 5-DoF Control of an Untethered Magnetic Millirobot via MRI Gradient Coils

Onder Erin, D. A. M. E. T., Sitti, M.

In IEEE International Conference on Robotics and Automation (ICRA), 2020 (inproceedings)

pi

[BibTex]

[BibTex]


Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis
Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesis

Liao, Y., Schwarz, K., Mescheder, L., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

Abstract
In recent years, Generative Adversarial Networks have achieved impressive results in photorealistic image synthesis. This progress nurtures hopes that one day the classical rendering pipeline can be replaced by efficient models that are learned directly from images. However, current image synthesis models operate in the 2D domain where disentangling 3D properties such as camera viewpoint or object pose is challenging. Furthermore, they lack an interpretable and controllable representation. Our key hypothesis is that the image generation process should be modeled in 3D space as the physical world surrounding us is intrinsically three-dimensional. We define the new task of 3D controllable image synthesis and propose an approach for solving it by reasoning both in 3D space and in the 2D image domain. We demonstrate that our model is able to disentangle latent 3D factors of simple multi-object scenes in an unsupervised fashion from raw images. Compared to pure 2D baselines, it allows for synthesizing scenes that are consistent wrt. changes in viewpoint or object pose. We further evaluate various 3D representations in terms of their usefulness for this challenging task.

avg

pdf suppmat Video 2 Project Page Video 1 Slides Poster [BibTex]

pdf suppmat Video 2 Project Page Video 1 Slides Poster [BibTex]


Exploring Data Aggregation in Policy Learning for Vision-based Urban Autonomous Driving
Exploring Data Aggregation in Policy Learning for Vision-based Urban Autonomous Driving

Prakash, A., Behl, A., Ohn-Bar, E., Chitta, K., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

Abstract
Data aggregation techniques can significantly improve vision-based policy learning within a training environment, e.g., learning to drive in a specific simulation condition. However, as on-policy data is sequentially sampled and added in an iterative manner, the policy can specialize and overfit to the training conditions. For real-world applications, it is useful for the learned policy to generalize to novel scenarios that differ from the training conditions. To improve policy learning while maintaining robustness when training end-to-end driving policies, we perform an extensive analysis of data aggregation techniques in the CARLA environment. We demonstrate how the majority of them have poor generalization performance, and develop a novel approach with empirically better generalization performance compared to existing techniques. Our two key ideas are (1) to sample critical states from the collected on-policy data based on the utility they provide to the learned policy in terms of driving behavior, and (2) to incorporate a replay buffer which progressively focuses on the high uncertainty regions of the policy's state distribution. We evaluate the proposed approach on the CARLA NoCrash benchmark, focusing on the most challenging driving scenarios with dense pedestrian and vehicle traffic. Our approach improves driving success rate by 16% over state-of-the-art, achieving 87% of the expert performance while also reducing the collision rate by an order of magnitude without the use of any additional modality, auxiliary tasks, architectural modifications or reward from the environment.

avg

pdf suppmat Video 2 Project Page Slides Video 1 [BibTex]

pdf suppmat Video 2 Project Page Slides Video 1 [BibTex]


no image
Fair Decisions Despite Imperfect Predictions

Kilbertus, N., Gomez Rodriguez, M., Schölkopf, B., Muandet, K., Valera, I.

Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), 2020 (conference) Accepted

ei plg

[BibTex]

[BibTex]


Learning Situational Driving
Learning Situational Driving

Ohn-Bar, E., Prakash, A., Behl, A., Chitta, K., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

Abstract
Human drivers have a remarkable ability to drive in diverse visual conditions and situations, e.g., from maneuvering in rainy, limited visibility conditions with no lane markings to turning in a busy intersection while yielding to pedestrians. In contrast, we find that state-of-the-art sensorimotor driving models struggle when encountering diverse settings with varying relationships between observation and action. To generalize when making decisions across diverse conditions, humans leverage multiple types of situation-specific reasoning and learning strategies. Motivated by this observation, we develop a framework for learning a situational driving policy that effectively captures reasoning under varying types of scenarios. Our key idea is to learn a mixture model with a set of policies that can capture multiple driving modes. We first optimize the mixture model through behavior cloning, and show it to result in significant gains in terms of driving performance in diverse conditions. We then refine the model by directly optimizing for the driving task itself, i.e., supervised with the navigation task reward. Our method is more scalable than methods assuming access to privileged information, e.g., perception labels, as it only assumes demonstration and reward-based supervision. We achieve over 98% success rate on the CARLA driving benchmark as well as state-of-the-art performance on a newly introduced generalization benchmark.

avg

pdf suppmat Video 2 Project Page Video 1 Slides [BibTex]

pdf suppmat Video 2 Project Page Video 1 Slides [BibTex]


On Joint Estimation of Pose, Geometry and svBRDF from a Handheld Scanner
On Joint Estimation of Pose, Geometry and svBRDF from a Handheld Scanner

Schmitt, C., Donne, S., Riegler, G., Koltun, V., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

Abstract
We propose a novel formulation for joint recovery of camera pose, object geometry and spatially-varying BRDF. The input to our approach is a sequence of RGB-D images captured by a mobile, hand-held scanner that actively illuminates the scene with point light sources. Compared to previous works that jointly estimate geometry and materials from a hand-held scanner, we formulate this problem using a single objective function that can be minimized using off-the-shelf gradient-based solvers. By integrating material clustering as a differentiable operation into the optimization process, we avoid pre-processing heuristics and demonstrate that our model is able to determine the correct number of specular materials independently. We provide a study on the importance of each component in our formulation and on the requirements of the initial geometry. We show that optimizing over the poses is crucial for accurately recovering fine details and that our approach naturally results in a semantically meaningful material segmentation.

avg

pdf Project Page Slides Video Poster [BibTex]

pdf Project Page Slides Video Poster [BibTex]


Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision
Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision

Niemeyer, M., Mescheder, L., Oechsle, M., Geiger, A.

In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2020, 2020 (inproceedings)

Abstract
Learning-based 3D reconstruction methods have shown impressive results. However, most methods require 3D supervision which is often hard to obtain for real-world datasets. Recently, several works have proposed differentiable rendering techniques to train reconstruction models from RGB images. Unfortunately, these approaches are currently restricted to voxel- and mesh-based representations, suffering from discretization or low resolution. In this work, we propose a differentiable rendering formulation for implicit shape and texture representations. Implicit representations have recently gained popularity as they represent shape and texture continuously. Our key insight is that depth gradients can be derived analytically using the concept of implicit differentiation. This allows us to learn implicit shape and texture representations directly from RGB images. We experimentally show that our single-view reconstructions rival those learned with full 3D supervision. Moreover, we find that our method can be used for multi-view 3D reconstruction, directly resulting in watertight meshes.

avg

pdf suppmat Video 2 Project Page Video 1 Video 3 Slides Poster [BibTex]

pdf suppmat Video 2 Project Page Video 1 Video 3 Slides Poster [BibTex]

2008


no image
Simulation and analysis of a passive pitch reversal flapping wing mechanism for an aerial robotic platform

Arabagi, V., Sitti, M.

In Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pages: 1260-1265, 2008 (inproceedings)

pi

Project Page [BibTex]

2008


Project Page [BibTex]


no image
Fabrication and Characterization of Biologically Inspired Mushroom-Shaped Elastomer Microfiber Arrays

Kim, S., Sitti, M.

In ASME 2008 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pages: 839-847, 2008 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Gecko inspired micro-fibrillar adhesives for wall climbing robots on micro/nanoscale rough surfaces

Aksak, B., Murphy, M. P., Sitti, M.

In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages: 3058-3063, 2008 (inproceedings)

pi

Project Page [BibTex]

Project Page [BibTex]


no image
Miniature Mobile Robots Down to Micron Scale

Sitti, M.

In Micro-NanoMechatronics and Human Science, 2008. MHS 2008. International Symposium on, pages: 525-525, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Polymeric Micro/Nanofiber Manufacturing and Mechanical Characterization

Nain, A. S., Sitti, M., Amon, C.

In ASME 2008 International Mechanical Engineering Congress and Exposition, pages: 295-303, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
An untethered magnetically actuated micro-robot capable of motion on arbitrary surfaces

Floyd, S., Pawashe, C., Sitti, M.

In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages: 419-424, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Fabrication of bio-inspired elastomer nanofiber arrays with spatulate tips using notching effect

Kim, S., Sitti, M., Jang, J., Thomas, E. L.

In Nanotechnology, 2008. NANO’08. 8th IEEE Conference on, pages: 780-782, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
A motorized anchoring mechanism for a tethered capsule robot using fibrillar adhesives for interventions in the esophagus

Glass, P., Cheung, E., Wang, H., Appasamy, R., Sitti, M.

In Biomedical Robotics and Biomechatronics, 2008. BioRob 2008. 2nd IEEE RAS & EMBS International Conference on, pages: 758-764, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Fabrication of Single and Multi-Layer Fibrous Biomaterial Scaffolds for Tissue Engineering

Nain, A. S., Miller, E., Sitti, M., Campbell, P., Amon, C.

In ASME 2008 International Mechanical Engineering Congress and Exposition, pages: 231-238, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Performance of different foot designs for a water running robot

Floyd, S., Adilak, S., Ramirez, S., Rogman, R., Sitti, M.

In Robotics and Automation, 2008. ICRA 2008. IEEE International Conference on, pages: 244-250, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Dynamic modeling of a basilisk lizard inspired quadruped robot running on water

Park, H. S., Floyd, S., Sitti, M.

In Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pages: 3101-3107, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Bacterial propulsion of chemically patterned micro-cylinders

Behkam, B., Sitti, M.

In Biomedical Robotics and Biomechatronics, 2008. BioRob 2008. 2nd IEEE RAS & EMBS International Conference on, pages: 753-757, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Design and Numerical Modeling of an On-Board Chemical Release Module for Motion Control of Bacteria-Propelled Swimming Micro-Robots

Behkam, B., Nain, A. S., Amon, C. H., Sitti, M.

In ASME 2008 International Mechanical Engineering Congress and Exposition, pages: 239-244, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Investigation of Calcium Mechanotransduction by Quasi 3-D Microfiber Mechanical Stimulation of Cells

Ruder, W. C., Pratt, E. D., Sitti, M., LeDuc, P. R., Antaki, J. F.

In ASME 2008 Summer Bioengineering Conference, pages: 1049-1050, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Beanbag robotics: Robotic swarms with 1-dof units

Kriesel, D. M., Cheung, E., Sitti, M., Lipson, H.

In International Conference on Ant Colony Optimization and Swarm Intelligence, pages: 267-274, 2008 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Particle image velocimetry and thrust of flagellar micro propulsion systems

Danis, U., Sitti, M., Pekkan, K.

In APS Division of Fluid Dynamics Meeting Abstracts, 1, 2008 (inproceedings)

pi

[BibTex]

[BibTex]

2001


no image
Survey of nanomanipulation systems

Sitti, M.

In Nanotechnology, 2001. IEEE-NANO 2001. Proceedings of the 2001 1st IEEE Conference on, pages: 75-80, 2001 (inproceedings)

pi

[BibTex]

2001


[BibTex]


no image
Nanotribological characterization system by AFM based controlled pushing

Sitti, M.

In Nanotechnology, 2001. IEEE-NANO 2001. Proceedings of the 2001 1st IEEE Conference on, pages: 99-104, 2001 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Towards flapping wing control for a micromechanical flying insect

Yan, J., Wood, R. J., Avadhanula, S., Sitti, M., Fearing, R. S.

In Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on, 4, pages: 3901-3908, 2001 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Man-machine interface for micro/nano manipulation with an afm probe

Aruk, B., Hashimoto, H., Sitti, M.

In Nanotechnology, 2001. IEEE-NANO 2001. Proceedings of the 2001 1st IEEE Conference on, pages: 151-156, 2001 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Development of PZT and PZN-PT based unimorph actuators for micromechanical flapping mechanisms

Sitti, M., Campolo, D., Yan, J., Fearing, R. S.

In Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on, 4, pages: 3839-3846, 2001 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Thorax Design and Wing Control for a Micromechanical Flying Insect

Yan, J, Ayadhanula, S, Sitti, M, Wood, RJ, Fearing, RS

In PROCEEDINGS OF THE ANNUAL ALLERTON CONFERENCE ON COMMUNICATION CONTROL AND COMPUTING, 39(2):952-961, 2001 (inproceedings)

pi

[BibTex]

[BibTex]


no image
PZT actuated four-bar mechanism with two flexible links for micromechanical flying insect thorax

Sitti, M.

In Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on, 4, pages: 3893-3900, 2001 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Development of a scaled teleoperation system for nano scale interaction and manipulation

Sitti, M., Aruk, B., Shintani, H., Hashimoto, H.

In Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on, 1, pages: 860-867, 2001 (inproceedings)

pi

[BibTex]

[BibTex]

1999


no image
Tele-touch feedback of surfaces at the micro/nano scale: Modeling and experiments

Sitti, M., Horighuchi, S., Hashimoto, H.

In Intelligent Robots and Systems, 1999. IROS’99. Proceedings. 1999 IEEE/RSJ International Conference on, 2, pages: 882-888, 1999 (inproceedings)

pi

[BibTex]

1999


[BibTex]


no image
Challenge to micro/nanomanipulation using atomic force microscope

Hashimoto, H., Sitti, M.

In Micromechatronics and Human Science, 1999. MHS’99. Proceedings of 1999 International Symposium on, pages: 35-42, 1999 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Visualization interface for AFM-based nano-manipulation

Horiguchi, S., Sitti, M., Hashimoto, H.

In Industrial Electronics, 1999. ISIE’99. Proceedings of the IEEE International Symposium on, 1, pages: 310-315, 1999 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Tele-nanorobotics 2-d manipulation of micro/nanoparticles using afm

Sitti, M., Horiguchi, S., Hashimoto, H.

In Advanced Intelligent Mechatronics, 1999. Proceedings. 1999 IEEE/ASME International Conference on, pages: 786-786, 1999 (inproceedings)

pi

[BibTex]

[BibTex]


no image
Two-dimensional fine particle positioning using a piezoresistive cantilever as a micro/nano-manipulator

Sitti, M., Hashimoto, H.

In Robotics and Automation, 1999. Proceedings. 1999 IEEE International Conference on, 4, pages: 2729-2735, 1999 (inproceedings)

pi

[BibTex]

[BibTex]