Unprocessing Seven Years of Algorithmic Fairness | Soziale Grundlagen der Informatik – Max Planck Institute for Intelligent Systems

Institute Homepage

Institute Homepage Sign In

Research Overview

Social Prediction

Performative Prediction: Past and Future

Difficult Lessons on Social Prediction from Wisconsin Public Schools

Allocation Requires Prediction Only if Inequality Is Low

Digital Platforms, Power and Work

Performative Power

An Engine Not a Camera: Measuring Performative Power of Online Search

Causal Inference from Competing Treatments

Contesting Algorithmic Systems

Algorithmic Collective Action in Machine Learning

Decline Now: A Combinatorial Model for Algorithmic Collective Action

Algorithmic Collective Action in Recommender Systems: Promoting Songs by Reordering Playlists

Algorithmic Fairness

Fairness and Machine Learning: Limitations and Opportunities

Unprocessing Seven Years of Algorithmic Fairness

Science of Machine Learning Benchmarks

ImageNot: A Contrast with ImageNet Preserves Model Rankings

Don't Label Twice: Quantity Beats Quality when Comparing Binary Classifiers on a Budget

A Theory of Dynamic Benchmarks

Evaluating Large Language Models

Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarks

Evaluating Language Models as Risk Scores

Fine-tuning Large Language Models

Training on the Test Task Confounds Evaluation and Emergence

Lawma: The Power of Specialization for Legal Tasks

Soziale Grundlagen der Informatik Members Publications

Unprocessing Seven Years of Algorithmic Fairness

Unprocessing3 — Example result showing the accuracy-fairness Pareto frontier achieved by postprocessing (dashed black line) contains all other methods.

This work takes a look at numerous algorithmic fairness methods published since 2016. After introducing a methodology, called unprocessing, that puts different methods on a level playing field, this computationally intensive meta study bears out a clear result: The simple “postprocessing” method of setting group specific thresholds works best.

Members

Thumb ticker sm andre innsbruck face 2

Soziale Grundlagen der Informatik

André Cruz

Doctoral Researcher

Thumb ticker sm 20241104 hardt moritz 12 cleaned kleiner

Soziale Grundlagen der Informatik

Moritz Hardt

Director

Publications

Social Foundations of Computation Conference Paper Unprocessing Seven Years of Algorithmic Fairness Cruz, A. F., Hardt, M. In The Twelfth International Conference on Learning Representations (ICLR 2024), May 2024 (Published) ArXiv Code URL BibTeX