Social Prediction
Nearly a hundred years ago, economist Oskar Morgenstern called it one of the most difficult problems in the theory of prediction. Unlike forecasts in astronomy, predictions in the social world can influence the outcome that they are trying to predict. Morgenstern was centrally concerned with economic forecasts, but his analysis applies equally to all social prediction problems of sufficient reach and scale. Predictions in the social world always influence what they aim to predict. As an example, consider a video streaming platform that orders content based on predicted watch time. The higher the predicted watch time, the higher up the video appears, thus increasing the likelihood that the user watches the video.
Predictions that influence the target of prediction we now call performative predictions. Together with Perdomo and Zrnic, Mendler-Dünner and Hardt co-founded the area of performative prediction a few years ago and have been key contributors since. Recently, they completed a first survey of the flourishing field of performative prediction and its many applications [].
While earlier work on performative prediction focused on optimization, more recent work turned to the problem of testing for performativity. Mendler-Dünner and co-authors developed effective statistical tests for performative effects based on causal inference methods []. In addition, work in the group proposed a model of a digital platform as a dynamic system and showed how to identify performative effects using the dynamic structure of the model [
].
In a major empirical evaluation of performativity, Perdomo, Britton, Hardt, and Abebe study risk scores used for early warning systems in the Wisconsin public education system []. A working early warning system should make self-negating prophecies, since it serves as a basis for positive interventions. Unfortunately, the early warning system studied here showed no effect despite high predictive accuracy. The results showed empirically that high predictive accuracy is not enough for a system to allocate scarce societal resources, such as interventions, effectively.
This insight kicked off a new research direction studying the interplay of prediction and allocation. At the heart of this new direction is a theoretical result showing that allocation requires accurate prediction only if inequality in the population is low and the planner’s budget is high []. In a related vein, we revisited Rawlsian social policy from the perspective of long-term social welfare, showing that in the long-run it can be surprisingly effective despite being less accurate than utilitarian policy [
].