Institute Homepage
Institute Homepage Sign In

Back

Empirical Inference Members Publications

teaching

teaching

Causal Representation Learning 

VVZ ETH Zurich

Deep neural networks have achieved impressive success on prediction tasks in a supervised learning setting, provided sufficient labelled data is available. However, current AI systems lack a versatile understanding of the world around us, as shown in a limited ability to transfer and generalize between tasks.

The course focuses on challenges and opportunities between deep learning and causal inference, and highlights work that attempts to develop statistical representation learning towards interventional/causal world models. The course will include guest lectures from renowned scientist both from academia as well as top industrial research labs.

The course covers amongst others the following topics:

  • Causal Inference and Causal Structure Learning
  • Deep Representation Learning 
  • Disentangled Representations
  • Independent Mechanisms
  • World Models and Interactive Learning

Grading 

The seminar is graded as pass/fail. In order to pass the course, participants need to write a summary of at least one lecture (n lectures if the team consists of n team members) and write reviews for at least two submissions (2*n reviews if the team consists of n team members). Course summaries and reviews will have to be submitted through openreview (link to be provided). For the assignment to a particular lecture a survey will be sent around. The course summary has to follow the NeurIPS format, however with 4 pages of text (references and appendix are limited to an additional 10 pages). Please find the template for the seminar below.

Seminar template: Download

The submission deadline: January 15, 2021.

Time and Place

Lectures

Tue, 16:00-18:00

 Online 

 

 

The zoom link for the online lectures will be send by email to registered students at ETH. If you have not done so please register for the course. 

 

Questions

If you have any questions, please use the Piazza group: piazza.com/ethz.ch/fall2020/263515500l

 

Please submit your lecture notes here: https://openreview.net/group?id=ETHz.ch/2020/Course/CRL

Lecture notes: Download (Disclaimer: These summaries were written by students have not been reviewed / proofread by the lecturers).

 

Syllabus

Day Lecture Topics Lecture Slides Recommended Reading Background Material
Sep 15 Introduction  Lecture 1 Elements of Causal Inference

Pearl, Judea, and Dana Mackenzie. The book of why: the new science of cause and effect. Basic Books, 2018.
Sep 22   Lecture 2 Elements of Causal Inference  
Sep 29

 

 Lecture 3 Elements of Causal Inference  
Oct 6                              Lecture 4 Guest: Sebastian Weichwald  
Oct 13                           Lecture 5 Guest: Francesco Locatello  
Oct 20   Lecture 6 Opening AI Center - no lecture  
Oct 27   Lecture 7 Guest: Ilya Tolstikhin  
Nov 3   Lecture 8 Guest: Irina Higgins  
Nov 10   Lecture 10 Guest: Patrick Schwab  
Nov 17   Lecture 11 Guest: Ferenc Huszár  
Nov 24   Lecture 12 Guest: Patrick Schwab  
Dec 1   Lecture 13 Guest: Anirudh Goyal  
Dec 20   Lecture 14 Guest: Silvia Chiappa  

 

Primary References

B. Schölkopf. "Causality for machine learning." arXiv preprint arXiv:1911.10500 (2019).

J. Peters,  D. Janzing, and B. Schölkopf. Elements of causal inference. The MIT Press, 2017.

Additional references

Pearl, Judea. Causality. Cambridge university press, 2009.

Hernán, Miguel A., and James M. Robins. "Causal inference: what if." Boca Raton: Chapman & Hill/CRC 2020 (2020).

Lectures at MLSS 2020

 

Assistants
 
 
Organizers
 
 
 
 
 
 
 
 
 
 

Members

Empirical Inference
Julius von Kügelgen
  • Doctoral Researcher
Empirical Inference
Lars Lorch
  • Doctoral Researcher
Empirical Inference
Bernhard Schölkopf
  • Director
Empirical Inference
Zhijing Jin
Research Scientist
Empirical Inference
Atalanti-Anastasia Mastakouri
Empirical Inference
Biwei Huang
Empirical Inference
Kun Zhang
Empirical Inference
Jonas Peters
Empirical Inference
Dominik Janzing
  • Research Scientist
Empirical Inference
Arash Mehrjou
Collaborator
Autonomous Learning
Maximilian Seitzer
  • Doctoral Researcher
no image
Empirical Inference
David Lopez-Paz
Empirical Inference
Mateo Rojas-Carulla
Empirical Inference
Mingming Gong
no image
Empirical Inference
Joris Mooij

Publications

Empirical Inference Conference Paper Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration Eastwood, C., Mason, I., Williams, C., Schölkopf, B. 10th International Conference on Learning Representations (ICLR), April 2022 (Published) arXiv URL BibTeX
Autonomous Learning Empirical Inference Conference Paper Causal Influence Detection for Improving Efficiency in Reinforcement Learning Seitzer, M., Schölkopf, B., Martius, G. In Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 34:22905-22918, (Editors: M. Ranzato and A. Beygelzimer and Y. Dauphin and P. S. Liang and J. Wortman Vaughan), Curran Associates, Inc., Red Hook, NY, 35th Conference on Neural Information Processing Systems, December 2021 (Published) arXiv PDF Data Code URL BibTeX
Empirical Inference Conference Paper DiBS: Differentiable Bayesian Structure Learning Lorch, L., Rothfuss, J., Schölkopf, B., Krause, A. In Advances in Neural Information Processing Systems 34, 29:24111-24123, (Editors: M. Ranzato and A. Beygelzimer and Y. Dauphin and P. S. Liang and J. Wortman Vaughan), Curran Associates, Inc., Red Hook, NY, 35th Annual Conference on Neural Information Processing Systems (NeurIPS 2021), December 2021 (Published) URL BibTeX
Empirical Inference Conference Paper Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP Jin*, Z., von Kügelgen*, J., Ni, J., Vaidhya, T., Kaushal, A., Sachan, M., Schölkopf, B. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 9499-9513, (Editors: Marie-Francine Moens and Xuanjing Huang and Lucia Specia and Scott Wen-tau Yih), Association for Computational Linguistics, November 2021, *equal contribution (Published) arXiv DOI URL BibTeX
Empirical Inference Conference Paper Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning Sontakke, S., Mehrjou, A., Itti, L., Schölkopf, B. Proceedings of 38th International Conference on Machine Learning (ICML), 139:9848-9858, Proceedings of Machine Learning Research, (Editors: Meila, Marina and Zhang, Tong), PMLR, July 2021 (Published) URL BibTeX
Empirical Inference Conference Paper Necessary and sufficient conditions for causal feature selection in time series with latent common causes Mastakouri, A. A., Schölkopf, B., Janzing, D. Proceedings of 38th International Conference on Machine Learning (ICML), 139:7502-7511, Proceedings of Machine Learning Research, (Editors: Meila, Marina and Zhang, Tong), PMLR, July 2021 (Published) arXiv URL BibTeX
Empirical Inference Article Simpson’s paradox in Covid-19 case fatality rates: a mediation analysis of age-related causal effects von Kügelgen*, J., Gresele*, L., Schölkopf, B. IEEE Transactions on Artificial Intelligence, 2(1):18-27, IEEE Computer Society, 2021, *equal contribution (Published) arXiv DOI URL BibTeX
Empirical Inference Conference Paper Causal analysis of Covid-19 Spread in Germany Mastakouri, A., Schölkopf, B. Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 3153-3163, (Editors: H. Larochelle and M. Ranzato and R. Hadsell and M. F. Balcan and H. Lin), Curran Associates, Inc., 34th Annual Conference on Neural Information Processing Systems, December 2020 (Published) URL BibTeX
Empirical Inference Conference Paper Semi-supervised learning, causality, and the conditional cluster assumption von Kügelgen, J., Mey, A., Loog, M., Schölkopf, B. Proceedings of the 36th International Conference on Uncertainty in Artificial Intelligence (UAI) , 124:1-10, Proceedings of Machine Learning Research, (Editors: Jonas Peters and David Sontag), PMLR, August 2020, *also at NeurIPS 2019 Workshop Do the right thing: machine learning and causal inference for improved decision making (Published) arXiv URL BibTeX
Empirical Inference Conference Paper A meta-transfer objective for learning to disentangle causal mechanisms Bengio, Y., Deleu, T., Rahaman, N., Ke, R., Lachapelle, S., Bilaniuk, O., Goyal, A., Pal, C. 8th International Conference on Learning Representations (ICLR), April 2020 (Published) arXiv URL BibTeX
Empirical Inference Article Causal Discovery from Heterogeneous/Nonstationary Data Huang, B., Zhang, K., Zhang, J., Ramsey, J., Sanchez-Romero, R., Glymour, C., Schölkopf, B. Journal of Machine Learning Research, 21(1):1-53, 2020 (Published) URL BibTeX
Empirical Inference Miscellaneous Learning Neural Causal Models from Unknown Interventions Ke, R., Bilaniuk, O., Goyal, A., Bauer, S., Larochelle, H., Schölkopf, B., Mozer, M. C., Pal, C., Bengio, Y. 2020 (Published) arXiv BibTeX
Empirical Inference Conference Paper Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features von Kügelgen, J., Mey, A., Loog, M. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS), 89:1361-1369, (Editors: Kamalika Chaudhuri and Masashi Sugiyama), PMLR, April 2019 (Published) PDF Poster URL BibTeX
Empirical Inference Article Inferring causation from time series with perspectives in Earth system sciences Runge, J., Bathiany, S., Bollt, E., Camps-Valls, G., Coumou, D., Deyle, E., Glymour, C., Kretschmer, M., Mahecha, M., Munoz-Mari, J., van Nes, E., Peters, J., Quax, R., Reichstein, M., Scheffer, M., Schölkopf, B., Spirtes, P., Sugihara, G., Sun, J., Zhang, K., et al. Nature Communications, 10(1):article no. 2553, 2019 (Published) DOI BibTeX
Empirical Inference Conference Paper Generalized Score Functions for Causal Discovery Huang, B., Zhang, K., Lin, Y., Schölkopf, B., Glymour, C. Proceedings of the 24th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 1551-1560, (Editors: Yike Guo and Faisal Farooq), ACM, August 2018 (Published) DOI URL BibTeX
Empirical Inference Conference Paper Cause-Effect Inference by Comparing Regression Errors Blöbaum, P., Janzing, D., Washio, T., Shimizu, S., Schölkopf, B. Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS) , 84:900-909, Proceedings of Machine Learning Research, (Editors: Amos Storkey and Fernando Perez-Cruz), PMLR, April 2018 (Published) URL BibTeX
Empirical Inference Article Invariant Models for Causal Transfer Learning Rojas-Carulla, M., Schölkopf, B., Turner, R., Peters, J. Journal of Machine Learning Research, 19(36):1-34, 2018 (Published) URL BibTeX
Empirical Inference Article Kernel-based tests for joint independence Pfister, N., Bühlmann, P., Schölkopf, B., Peters, J. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(1):5-31, 2018 (Published) DOI BibTeX
Empirical Inference Conference Paper Behind Distribution Shift: Mining Driving Forces of Changes and Causal Arrows Huang, B., Zhang, K., Zhang, J., Sanchez-Romero, R., Glymour, C., Schölkopf, B. IEEE 17th International Conference on Data Mining (ICDM), 913-918, (Editors: Vijay Raghavan,Srinivas Aluru, George Karypis, Lucio Miele and Xindong Wu), November 2017 (Published) DOI BibTeX
Empirical Inference Conference Paper Causal Discovery from Nonstationary/Heterogeneous Data: Skeleton Estimation and Orientation Determination Zhang, K., Huang, B., Zhang, J., Glymour, C., Schölkopf, B. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI), 1347-1353, (Editors: Carles Sierra), August 2017 (Published) PDF DOI BibTeX
Empirical Inference Conference Paper Discovering Causal Signals in Images Lopez-Paz, D., Nishihara, R., Chintala, S., Schölkopf, B., Bottou, L. Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017, 58-66, IEEE, Piscataway, NJ, USA, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017 (Published) DOI URL BibTeX
Empirical Inference Book Elements of Causal Inference - Foundations and Learning Algorithms Peters, J., Janzing, D., Schölkopf, B. Adaptive Computation and Machine Learning Series, The MIT Press, Cambridge, MA, USA, 2017 (Published) PDF URL BibTeX
Empirical Inference Conference Paper Domain Adaptation with Conditional Transferable Components Gong, M., Zhang, K., Liu, T., Tao, D., Glymour, C., Schölkopf, B. Proceedings of the 33nd International Conference on Machine Learning (ICML), 48:2839-2848, JMLR Workshop and Conference Proceedings, (Editors: Balcan, M.-F. and Weinberger, K. Q.), June 2016 (Published) URL BibTeX
Empirical Inference Conference Paper Learning Network of Multivariate Hawkes Processes: A Time Series Approach Etesami, S., Kiyavash, N., Zhang, K., Singhal, K. Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (UAI), 162-171, (Editors: Ihler, Alexander T. and Janzing, Dominik), June 2016 (Published) URL BibTeX
Empirical Inference Conference Paper On the Identifiability and Estimation of Functional Causal Models in the Presence of Outcome-Dependent Selection Zhang, K., Zhang, J., Huang, B., Schölkopf, B., Glymour, C. Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (UAI), 825-834, (Editors: Ihler, Alexander T. and Janzing, Dominik), June 2016 (Published) URL BibTeX
Empirical Inference Conference Paper The Arrow of Time in Multivariate Time Serie Bauer, S., Schölkopf, B., Peters, J. Proceedings of the 33rd International Conference on Machine Learning (ICML), 48:2043-2051, JMLR Workshop and Conference Proceedings, (Editors: Balcan, M. F. and Weinberger, K. Q.), JMLR, June 2016 (Published) URL BibTeX
Empirical Inference Article A Causal, Data-driven Approach to Modeling the Kepler Data Wang, D., Hogg, D. W., Foreman-Mackey, D., Schölkopf, B. Publications of the Astronomical Society of the Pacific, 128(967):094503, 2016, Astrophysics Source Code Library ascl: 2107.024 (Published) DOI URL BibTeX
Empirical Inference Article Algorithmic independence of initial condition and dynamical law in thermodynamics and causal inference Janzing, D., Chaves, R., Schölkopf, B. New Journal of Physics, 18(9):article no. 093052, 2016 (Published) PDF DOI URL BibTeX
Empirical Inference Article Causal inference using invariant prediction: identification and confidence intervals Peters, J., Bühlmann, P., Meinshausen, N. Journal of the Royal Statistical Society, Series B (Statistical Methodology), 78(5):947-1012, 2016, (with discussion) (Published) DOI URL BibTeX
Empirical Inference Article Distinguishing cause from effect using observational data: methods and benchmarks Mooij, J., Peters, J., Janzing, D., Zscheischler, J., Schölkopf, B. Journal of Machine Learning Research, 17(32):1-102, 2016 (Published) URL BibTeX
Empirical Inference Article Modeling Confounding by Half-Sibling Regression Schölkopf, B., Hogg, D., Wang, D., Foreman-Mackey, D., Janzing, D., Simon-Gabriel, C. J., Peters, J. Proceedings of the National Academy of Science, 113(27):7391-7398, 2016 (Published) Code DOI URL BibTeX
Empirical Inference Book Chapter Nonlinear functional causal models for distinguishing cause from effect Zhang, K., Hyvärinen, A. In Statistics and Causality: Methods for Applied Empirical Research, 185-201, 8, 1st, (Editors: Wolfgang Wiedermann and Alexander von Eye), John Wiley & Sons, Inc., 2016 (Published) BibTeX