Counterfactual Token Generation in Large Language Models

In this talk, I will begin by illustrating why the stateless nature of state-of-the-art large language models—i.e., their lack of internal memory or state—prevents us from answering such counterfactual questions. To address this limitation, I will introduce a causal model of token generation based on the Gumbel-Max structural causal model, along with a method that allows us to equip an arbitrary large language model with the ability to perform counterfactual token generation. I will then share experimental results from implementing this approach on Llama 3 8B-Instruct and Ministral-8B-Instruct, highlighting both qualitative and quantitative insights about counterfactual text. Finally, I will conclude with a demonstrative application of counterfactual token generation for bias detection, unveiling interesting insights about the (social) model of the world constructed by large language models.