Empirical Inference
Talk
Professor Jürgen Schmidhuber
11-04-2019
Unsupervised Learning: Passiv and Active
I’ll start with a concept of 1990 that has become popular: unsupervised learning without a teacher through two adversarial neural networks (NNs) that duel in a minimax game, where one NN minimizes the objective function maximized by the other. The first NN generates data through its output actions, the second NN predicts the data. The second NN minimizes its error, thus becoming a better predictor. But it is a zero sum game: the first NN tries to find actions that maximize the error of the second NN. The system exhibits what I called “artificial curiosity” because the first NN is motivated ...
Bernhard Schölkopf
Empirical Inference
IS Colloquium
Benjamin Bloem-Reddy
25-03-2019
Probabilistic symmetry and invariant neural networks
In an effort to improve the performance of deep neural networks in data-scarce, non-i.i.d., or unsupervised settings, much recent research has been devoted to encoding invariance under symmetry transformations into neural network architectures. We treat the neural network input and output as random variables, and consider group invariance from the perspective of probabilistic symmetry. Drawing on tools from probability and statistics, we establish a link between functional and probabilistic symmetry, and obtain functional representations of probability distributions that are invariant or eq...
Isabel Valera
Empirical Inference
Talk
Prof. Martin Spindler
11-06-2018
Double Machine Learning with two Applications: Transformation Models and Gaussian Graphical Models in High-Dimensional Settings
In this talk first an introduction to the double machine learning framework is given. This allows inference on parameters in high-dimensional settings. Then, two applications are given, namely transformation models and Gaussian graphical models in high-dimensional settings. Both kind of models are widely used by practitioners. As high-dimensional data sets become more and more available, it is important to allow situations where the number of parameters is large compared to the sample size.
Philipp Geiger
Empirical Inference
IS Colloquium
Cédric Archambeau
11-06-2018
Learning Representations for Hyperparameter Transfer Learning
Bayesian optimization (BO) is a model-based approach for gradient-free black-box function optimization, such as hyperparameter optimization. Typically, BO relies on conventional Gaussian process regression, whose algorithmic complexity is cubic in the number of evaluations. As a result, Gaussian process-based BO cannot leverage large numbers of past function evaluations, for example, to warm-start related BO runs. After a brief intro to BO and an overview of several use cases at Amazon, I will discuss a multi-task adaptive Bayesian linear regression model, whose computational complexity is ...
Isabel Valera
Empirical Inference
Talk
Emily BJ Coffey
14-05-2018
Machine learning in cognitive neuroscience: questions, challenges and potential opportunities
In this talk I will describe the main types of research questions and neuroimaging tools used in my work in human cognitive neuroscience (with foci in audition and sleep), some of the existing approaches used to analyze our data, and their limitations. I will then discuss the main practical obstacles to applying machine learning methods in our field. Several of my ongoing and planned projects include research questions that could be addressed and perhaps considerably extended using machine learning approaches; I will describe some specific datasets and problems, with the goal of exploring i...
Mara Cascianelli
Empirical Inference
IS Colloquium
Patrick Bajari
23-03-2018
The Impact of Big Data on Firm Performance: an Empirical Investigation
In academic and policy circles, there has been considerable interest in the impact of “big data” on firm performance. We examine the question of how the amount of data impacts the accuracy of Machine Learned models of weekly retail product forecasts using a proprietary data set obtained from Amazon. We examine the accuracy of forecasts in two relevant dimensions: the number of products (N), and the number of time periods for which a product is available for sale (T). Theory suggests diminishing returns to larger N and T, with relative forecast errors diminishing at rate 1/sqrt(N) + 1/sqrt(...
Michel Besserve
Michael Hirsch
Empirical Inference
IS Colloquium
Bin Yu
05-03-2018
Three principles of data science: predictability, stability, and computability
In this talk, I'd like to discuss the intertwining importance and connections of three principles of data science in the title. They will be demonstrated in the context of two collaborative projects in neuroscience and genomics, respectively. The first project in neuroscience
uses transfer learning to integrate fitted convolutional neural networks (CNNs) on ImageNet with regression methods to provide predictive and stable characterizations of neurons from the challenging primary visual cortex V4. The second project proposes iterative random forests (iRF) as a stablized RF to seek predictab...
Michel Besserve
Empirical Inference
Talk
Aljoscha Leonhardt
27-02-2018
A naturalistic perspective on optic flow processing in the fly
Optic flow offers a rich source of information about an organism’s environment. Flies, for instance, are thought to make use of motion vision to control and stabilise their course during acrobatic airborne manoeuvres. How these computations are implemented in neural hardware and how such circuits cope with the visual complexity of natural scenes, however, remain open questions. This talk outlines some of the progress we have made in unraveling the computational substrate underlying optic flow processing in Drosophila. In particular, I will focus on our efforts to connect neural mechanisms a...
Michel Besserve
Empirical Inference
IS Colloquium
Simon Lacoste-Julien
23-10-2017
Modern Optimization for Structured Machine Learning
Machine learning has become a popular application domain for modern optimization techniques, pushing its algorithmic frontier. The need for large scale optimization algorithms which can handle millions of dimensions or data points, typical for the big data era, have brought a resurgence of interest for first order algorithms, making us revisit the venerable stochastic gradient method [Robbins-Monro 1951] as well as the Frank-Wolfe algorithm [Frank-Wolfe 1956]. In this talk, I will review recent improvements on these algorithms which can exploit the structure of modern machine learning appro...
Philipp Hennig
Empirical Inference
IS Colloquium
Dominik Bach
02-10-2017
Algorithms for survival: a decision-theoretic perspective on adaptive action under threat
Under acute threat, biological agents need to choose adaptive actions to survive. In my talk, I will provide a decision-theoretic view on this problem and ask, what are potential computational algorithms for this choice, and how are they implemented in neural circuits. Rational design principles and non-human animal data tentatively suggest a specific architecture that heavily relies on tailored algorithms for specific threat scenarios. Virtual reality computer games provide an opportunity to translate non-human animal tasks to humans and investigate these algorithms across species. I will ...
Michel Besserve
Empirical Inference
Talk
Ioannis Papantonis
24-07-2017
Adaptive Learning Rate Algorithms for Stochastic Optimization and Variational Bayesian Inference
We present a way to set the step size of Stochastic Gradient Descent, as the solution of
a distance minimization problem. The obtained result has an intuitive interpretation and
resembles the update rules of well known optimization algorithms. Also, asymptotic results
to its relation to the optimal learning rate of Gradient Descent are discussed.
In addition, we talk about two different estimators, with applications in
Variational inference problems, and present approximate results about their variance.
Finally, we combine all of the above, to present an optimization algorithm that...
Philipp Hennig
Empirical Inference
Talk
Prof. Stéphanie Lacour
11-07-2017
Soft bioelectronics: Materials and Technology
Bioelectronics integrates principles of electrical engineering and materials science to biology, medicine and ultimately health. Soft bioelectronics focus on designing and manufacturing electronic devices with mechanical properties close to those of the host biological tissue so that long-term reliability and minimal perturbation are induced in vivo and/or truly wearable systems become possible. We illustrate the potential of this soft technology with examples ranging from prosthetic tactile skins to soft multimodal neural implants.
Empirical Inference
Talk
Chris Bauch
10-07-2017
Sentiment analysis of tweets to detect tipping points in vaccinating behaviour
Vaccine refusal can lead to outbreaks of previously eradicated diseases and is an increasing problem worldwide. Vaccinating decisions exemplify a complex, coupled system where vaccinating behavior and disease dynamics influence one another. Complex systems often exhibit characteristic dynamics near a tipping point to a new dynamical regime. For instance, critical slowing down -- the tendency for a system to start `wobbling'-- can increase close to a tipping point. We used a linear support vector machine to classify the sentiment of geo-located United States and California tweets concern...
Empirical Inference
IS Colloquium
Frederick Eberhardt
03-07-2017
Causal Macro Variables
Standard methods of causal discovery take as input a statistical data set of
measurements of well-defined causal variables. The goal is then to determine
the causal relations among these variables. But how are these causal
variables identified or constructed in the first place? Often we have sensor
level data but assume that the relevant causal interactions occur at a
higher scale of aggregation. Sometimes we only have aggregate measurements
of causal interactions at a finer scale. I will motivate the general problem
of causal discovery and present recent work on a framework and meth...
Sebastian Weichwald
Empirical Inference
Talk
Felix Leibfried and Jordi Grau-Moya
13-06-2017
Model-based reinforcement learning for sequential decision-making
Autonomous systems rely on learning from experience to automatically refine their strategy and adapt to their environment, and thereby have huge advantages over traditional hand engineered systems. At PROWLER.io we use reinforcement learning (RL) for sequential decision making under uncertainty to develop intelligent agents capable of acting in dynamic and unknown environments. In this talk we first give a general overview of the goals and the research conducted at PROWLER.io. Then, we will talk about two specific research topics. The first is Information-Theoretic Model Uncertainty which d...
Michel Besserve
Empirical Inference
IS Colloquium
Sebastian Nowozin
29-05-2017
Probabilistic Deep Learning: From Density Estimation to Representation Learning
Probabilistic deep learning methods have recently made great progress
for generative and discriminative modeling. I will give a brief overview of
recent developments and then present two contributions.
The first is on a generalization of generative adversarial networks (GAN),
extending their use considerably. GANs can be shown to approximately minimize
the Jensen-Shannon divergence between two distributions, the true sampling
distribution and the model distribution. We extend GANs to the class of
f-divergences which include popular divergences such as the Kullback-Leibler
diver...
Lars Mescheder
Empirical Inference
IS Colloquium
John Cunningham
06-03-2017
Statistical testing of epiphenomena for multi-index data
As large tensor-variate data increasingly become the norm in applied machine learning and statistics, complex analysis methods similarly increase in prevalence. Such a trend offers the opportunity to understand more intricate features of the data that, ostensibly, could not be studied with simpler datasets or simpler methodologies. While promising, these advances are also perilous: these novel analysis techniques do not always consider the possibility that their results are in fact an expected consequence of some simpler, already-known feature of simpler data (for example, treating the te...
Philipp Hennig
Empirical Inference
IS Colloquium
Fabien Lotte
19-12-2016
Human Learning and Alternative Applications Towards Usable Electroencephalography-based Brain-Computer Interfaces
Brain-Computer Interfaces (BCIs) are systems that can translate brain activity patterns of a user into messages or commands for an interactive application. Such brain activity is typically measured using Electroencephalography (EEG), before being processed and classified by the system. EEG-based BCIs have proven promising for a wide range of applications ranging from communication and control for motor impaired users, to gaming targeted at the general public, real-time mental state monitoring and stroke rehabilitation, to name a few. Despite this promising potential, BCIs are still scarcely...
Empirical Inference
Talk
Sascha Quantz
01-03-2016
Images of planets orbiting other stars
The detection and characterization of planets orbiting other stars than the Sun, i.e., so-called extrasolar planets, is one of the fastest growing and most vibrant research fields in modern astrophysics. In the last 25 years, more than 5400 extrasolar planets and planet candidates were revealed, but the vast majority of these objects was detected with indirect techniques, where the existence of the planet is inferred from periodic changes in the light coming from the central star. No photons from the planets themselves are detected. In this talk, however, I will focus on the direct detectio...
Empirical Inference
IS Colloquium
Aldo Faisal
25-01-2016
What can we learn if we record the complete perception and action of a person?
Our research questions are centred on a basic characteristic of human brains: variability in their behaviour and their underlying meaning for cognitive mechanisms. Such variability is emerging as a key ingredient in understanding biological principles (Faisal, Selen & Wolpert, 2008, Nature Rev Neurosci) and yet lacks adequate quantitative and computational methods for description and analysis. Crucially, we find that biological and behavioural variability contains important information that our brain and our technology can make us of (instead of just averaging it away): Using advanced body...
Matthias Hohmann
Empirical Inference
IS Colloquium
Jonas Richiardi
19-10-2015
Imaging genomics of functional brain networks
During rest, brain activity is intrinsically synchronized between different brain regions, forming networks of coherent activity. These functional networks (FNs), consisting of multiple regions widely distributed across lobes and hemispheres, appear to be a fundamental theme of neural organization in mammalian brains.
Despite hundreds of studies detailing this phenomenon, the genetic and molecular mechanisms supporting these functional networks remain undefined. Previous work has mostly focused on polymorphisms in candidate genes, or used a twin study approach to demonstrate heritability...
Moritz Grosse-Wentrup
Michel Besserve
Empirical Inference
IS Colloquium
Sach Mukherjee
28-09-2015
High-dimensional statistical approaches for personalized medicine
Human diseases show considerable heterogeneity at the molecular level. Such heterogeneity is central to personalized medicine efforts that seek to exploit molecular data to better understand disease biology and inform clinical decision making. An emerging notion is that diseases and disease subgroups may differ not only at the level of mean molecular abundance, but also with respect to patterns of molecular interplay. I will discuss our ongoing efforts to develop methods to investigate such heterogeneity, with an emphasis on some high-dimensional aspects.
Michel Besserve
Jonas Peters
Empirical Inference
IS Colloquium
Kevin T. Kelly
27-07-2015
New Foundations for Ockham’s Razor in Statistical Inductive Inference, with Applications to Causal Discovery from Non-experimental Data
In machine learning, the standard explanation of Ockham's razor is to minimize predictive risk. But prediction is interpreted passively---one may not rely on predictions to change the probability distribution used for training. That limitation may be overcome by studying alternatively manipulated systems in randomized experimental trials, but experiments on multivariate systems or on human subjects are often infeasible or immoral. Happily, the past three decades have witnessed the development of a range of statistical techniques for discovering causal relations from non-experimental dat...
Michel Besserve
Kun Zhang
Empirical Inference
Conference
13-07-2015
- 24-07-2015
MLSS 2015 - Machine Learning Summer School 2015 in Tübingen
MLSS's are a renowned venue for graduate students, researchers, and professionals. They offer an opportunity to learn about fundamental and advanced aspects of machine learning, data analysis and inference, from intellectual leaders of the field.
<br>
In the summer of 2015 the MLSS will make its fourth appearance in Tübingen.
Michael Hirsch
Philipp Hennig
Bernhard Schölkopf
Empirical Inference
IS Colloquium
Joaquin Quiñonero Candela
11-08-2014
Examples of Machine Learning and Data Science at Facebook
Facebook serves close to a billion people every day, who are only able to consume a small subset of the information available to them. In this talk I will give some examples of how machine learning is used to personalize people’s Facebook experience. I will also present some data science experiments with fairly counter-intuitive results.
Empirical Inference
IS Colloquium
Manfred Opper
15-07-2014
Approximate inference for stochastic differential equations
Stochastic differential equations (SDEs) arise naturally as descriptions of continuous time dynamical systems. My talk addresses the problem of inferring the dynamical state and parameters of such systems from observations taken at discrete times. I will discuss the application of approximate inference methods such as the variational method and expectation propagation and show how higher dimensional systems can be treated by a mean field approximation.
In the second part of my talk I will discuss the nonparametric estimation of the drift (i.e. the deterministic part of the ‘force’ whi...
Philipp Hennig
Michel Besserve
Empirical Inference
IS Colloquium
Holger Rauhut
23-06-2014
Compressive Sensing and Beyond
The recent theory of compressive sensing predicts that (approximately) sparse vectors can be recovered from vastly incomplete linear measurements using efficient algorithms.
This principle has a large number of potential applications in signal and image processing, machine learning and more. Optimal measurement matrices in this context known so far are based on randomness. Recovery algorithms include convex optimization approaches (l1-minimization) as well as greedy methods. Gaussian and Bernoulli random matrices are provably optimal in the sense that the smallest possible number of...
Michel Besserve
Empirical Inference
Symposium
04-06-2014
- 06-06-2014
IEEE Conference on Pattern Recognition in Neuroimaging, Tübingen 2014
Multivariate analysis of neuroimaging data has gained ground very rapidly in the community over the past few years, leading to impressive results in cognitive and clinical neuroscience. Pattern recognition and machine learning conferences regularly feature a neuroimaging workshop, while neuroscientific meetings dedicate sessions to new approaches to neural data analysis. Thus, a rich two-way flow has been established between disciplines. It is the goal of the PRNI workshop series to continue facilitating exchange of ideas between scientific communities, with a particular interest in new app...
Moritz Grosse-Wentrup
Empirical Inference
IS Colloquium
Christian Lubich
26-05-2014
Low-rank dynamics
This talk reviews differential equations on manifolds of matrices or tensors of low rank. They serve to approximate, in a low-rank format, large time-dependent matrices and tensors that are either given explicitly via their increments or are unknown solutions of differential equations. Furthermore, low-rank differential equations are used in novel algorithms for eigenvalue optimisation, for instance in robust-stability problems.
Philipp Hennig
Empirical Inference
IS Colloquium
Simo Särkkä
07-04-2014
State-space representation of gaussian processes for regression and efficient inference in latent force model
Gaussian process regression is a non-parametric Bayesian machine learning paradigm, where instead of estimating parameters of fixed-form functions, we model the whole unknown functions as Gaussian processes. Gaussian processes are also commonly used for representing uncertainties in models of dynamic systems in many applications such as tracking, navigation, and automatic control systems. The latter models are often formulated as state-space models, where the use of non-linear Kalman filter type of methods is common. The aim of this talk is to discuss connections of Kalman filtering method...
Philipp Hennig
Empirical Inference
IS Colloquium
Rainer Dahlhaus
31-03-2014
Online Spot Volatility-Estimation and Decomposition with Nonlinear Market Microstructure Noise Models
(joint work with Jan. C. Neddermeyer)
A technique for online estimation of spot volatility for high-frequency data is developed. The algorithm works directly on the transaction data and updates the volatility estimate immediately after the occurrence of a new transaction. Furthermore, a nonlinear market microstructure noise model is proposed that reproduces several stylized facts of high frequency data. A computationally efficient particle filter is used that allows for the approximation of the unknown efficient prices and, in combination with a recursive EM algorithm, for the estimation ...
Michel Besserve
Empirical Inference
IS Colloquium
Peter Battaglia
28-03-2014
Simulation in physical scene understanding
Our ability to understand a scene is central to how we interact with our environment and with each other. Classic research on visual scene perception has focused on how people "know what is where by looking", but this talk will explore people's ability to infer the "hows" and "whys" of their world, and in particular, how they form a physical understanding of a scene. From a glance we can know so much: not only what objects are where, but whether they are movable, fragile, slimy, or hot; whether they were made by hand, by machine, or by nature; whether they are broken and how they could be r...
Michel Besserve