Institute Homepage
Institute Homepage DE Sign In

Events & Talks

Deep Models and Optimization Talk Upcoming 28-11-2025 How does data shape learning in LLMs? A case study of factual recall and the surprising role of data diversity (by Nicolas Zucchet, ETH Zurich) Data drives LLM training, yet we have limited scientific understanding of how it shapes learning dynamics and thus the final model. This talk, based on two recent works [1 <https://arxiv.org/abs/2503.21676> ][2 <https://arxiv.org/abs/2505.17863> ] will examine these questions with a focus on factual recall. We will begin by analyzing how LLMs learn a synthetic factual recall task, that serves as a test bed for knowledge acquisition and where we can precisely control data distribution properties. Our experiments reveal that learning proceeds in distinct stages, and, surprisingly, that skewe... Antonio Orvieto
Thumb ticker sm photo nicolas zucchet