Deep learning has significantly advanced state-of-the-art for 3D hand pose estimation, of which accuracy can be improved with increased amounts of labelled data. However, acquiring 3D hand pose labels can be extremely difficult. In this talk, I will present our recent two works on leveraging self-supervised learning techniques for hand pose estimation from depth map. In both works, we incorporate differentiable renderer to the network and formulate training loss as model fitting error to update network parameters. In first part of the talk, I will present our earlier work which approximates hand surface with a set of spheres. We then model the pose prior as a variational lower bound with variational auto-encoder(VAE). In second part, I will present our latest work on regressing the vertex coordinates of a hand mesh model with 2D fully convolutional network(FCN) in a single forward pass. In the first stage, the network estimates a dense correspondence field for every pixel on the image grid to the mesh grid. In the second stage, we design a differentiable operator to map features learned from the previous stage and regress a 3D coordinate map on the mesh grid. Finally, we sample from the mesh grid to recover the mesh vertices, and fit it an articulated template mesh in closed form. Without any human annotation, both works can perform competitively with strongly supervised methods. The later work will also be later extended to be compatible with MANO model.
Organizers: Dimitrios Tzionas
Robots today rely on rigid components and electric motors based on metal and magnets, making them heavy, unsafe near humans, expensive and ill-suited for unpredictable environments. Nature, in contrast, makes extensive use of soft materials and has produced organisms that drastically outperform robots in terms of agility, dexterity, and adaptability. The Keplinger Lab aims to fundamentally challenge current limitations of robotic hardware, using an interdisciplinary approach that synergizes concepts from soft matter physics and chemistry with advanced engineering technologies to introduce robotic materials – material systems that integrate actuation, sensing and even computation – for a new generation of intelligent systems. This talk gives an overview of fundamental research questions that inspire current and future research directions. One major theme of research is the development of new classes of actuators – a key component of all robotic systems – that replicate the sweeping success of biological muscle, a masterpiece of evolution featuring astonishing all-around actuation performance, the ability to self-heal after damage, and seamless integration with sensing. A second theme of research are functional polymers with unusual combinations of properties, such as electrical conductivity paired with stretchability, transparency, biocompatibility and the ability to self-healing from mechanical and electrical damage. A third theme of research is the discovery of new energy capture principles that can provide power to intelligent autonomous systems, as well as – on larger scales – enable sustainable solutions for the use of waste heat from industrial processes or the use of untapped sources of renewable energy, such as ocean waves.
Magnetic fields and light can be used to assemble, manipulate, and heat nanoparticles (NPs) and to remotely actuate polymer composites. Simple soft robots will be presented, where incorporation of magnetic and plasmonic NPs makes them responsive to magnetic fields and light. Application of magnetic fields to dispersions of magnetic NPs drives their assembly into chains. Dipolar coupling within the chains is a source of magnetic anisotropy, and chains of magnetic NPs embedded in a polymer matrix can be used to program the response of soft robots, while still using simple architectures. Wavelength-selective photothermal triggering of shape recovery in shape memory polymers with embedded Au nanospheres and nanorods can be used to remotely drive sequential processes. Combining magnetic actuation and photothermal heating enables remote configuration, locking, unlocking, and reconfiguration of soft robots, thus increasing their capabilities. Composite and multifunctional NPs are of interest for expanding the properties and applications of NPs. Silica shells are desirable for facilitating functionalization with silanes and enhancing the stability of NPs. Methods for depositing thin silica shells with controlled morphologies onto Au nanorods and CdSe/CdS core/shell quantum dot nanorods will be presented. Silica deposition can also be accompanied by etching and breakage of the core NPs. Assembly of Fe3O4 NPs onto silica-overcoated Au nanorods allows for magnetic manipulation, while retaining the surface plasmon resonance.
Organizers: Metin Sitti
Realistic digital avatars are increasingly important in digital media with potential to revolutionize 3D face-to-face communication and social interactions through compelling digital embodiment of ourselves. My goal is to efficiently create high-fidelity 3D avatars from a single image input, captured in an unconstrained environment. These avatars must be close in quality to those created by professional capture systems, yet require minimal computation and no special expertise from the user. These requirements pose several significant technical challenges. A single photograph provides only partial information due to occlusions, and intricate variations in shape and appearance may prevent us from applying traditional template-based approaches. In this talk, I will present our recent work on clothed human reconstruction from a single image. We demonstrate that a careful choice of data representation that can be easily handled by machine learning algorithms is the key to robust and high-fidelity synthesis and inference for human digitization.
Organizers: Timo Bolkart
Fraunhofer IPA in Stuttgart is one of the largest institutes within the Fraunhofer Society with a strong focus on production technologies and automation. Research and technology transfer efforts on machine learning and artificial intelligence are concentrated at IPA’s Center for Cyber Cognitive Intelligence (CCI). This talk gives an introduction to CCI‘s mission and typical industrial applications being addressed. Furthermore, an overview of the research areas and a deep dive into selected topics are provided. Examples are 6D pose estimation for robotic bin-picking, explainable machine learning, Bayesian filtering for object tracking, or event correlation mining.
Organizers: Sebastian Trimpe
Where does causal thinking meet machine learning? We will discuss several such cases. We first show how we use learning theory to guide us in building algorithms for inferring individual-level causal effects, and how we apply these ideas to create deep-learning causal-effect inference methods. We then show how ideas from causal inference can help us in two important machine learning tasks: learning robust classifiers and interpreting deep image recognition system. If time permits, we’ll discuss a recent application of machine learning for learning individualized treatments for patients in an acute hospital setting.
Organizers: Krikamol Muandet
Reducing the size and emissions of gas turbine engines used in the aeronautics industry forces manufacturers to explore new operating conditions. An undesirable phenomenon called thermo-acoustic instabilities may occur, caused by the coupling between combustion dynamics and the acoustics of the combustion chamber. To help predict, detect and suppress it, we explore various approaches. We will discuss the design of observers for infinite-dimensional systems, Fourier-based reduced-order modeling as well as a Machine-Learning approach based on high-fidelity simulation data.
Geometry is concerned with the properties of configurations of points, lines, and circles, while topology is concerned with space, dimension, and transformation. Geometry is also materials independent and scale invariant. By introducing holes and cuts in 2D sheets, we demonstrate dramatic shape change and super-conformability via expanding or collapsing of the hole arrays without deforming individual lattice units. When choosing the cuts and geometry correctly, we show folding into the third dimension, known as kirigami. The kirigami structures can be rendered pluripotent, that is changing into different 3D structures from the same 2D sheet. We explore their potential applications in energy efficient building facade, super-stretchable and shape conformable energy storage devices and medical devices, as well as bioinspired robotics. Programmable shape-shifting materials can take different physical forms to achieve multifunctionality in a dynamic and controllable manner. Through designs of geometric surface patterns, e.g. microchannels, we program the orientational elasticity in liquid crystal elastomers (LCEs), to direct folding of the 2D sheets into 3D shapes, which can be triggered by heat, light, and electric field. Taking this knowledge of guided inhomogeneous local deformations in LCEs, we then tackle the inverse problem – pre-programming geometry on a flat sheet to take an arbitrary desired 3D shape. Lastly, I will show the prospective of taking geometry to create smart fabrics and tendon-like filaments for soft robotic applications.
We develop a grammatical error correction system for German using a small gold corpus augmented with edits extracted from Wikipedia revision history. We extend the automatic error annotation tool ERRANT (Bryant et al., 2017) for German and use it to analyze both gold corrections and Wikipedia edits (Grundkiewicz and Junczys-Dowmunt, 2014) in order to select as additional training data Wikipedia edits containing grammatical corrections similar to those in the gold corpus. Using a neural machine translation approach (Chollampatt and Ng, 2018), we evaluate the contribution of Wikipedia edits and find that carefully selected Wikipedia edits increase performance by over 5%.
Organizers: Jean-Claude Passy