Jonas is a ML researcher in Tübingen, Germany, where he leads the research group for safety- & efficiency- aligned learning. He is a Max-Planck research group leader and Hector-Endowed Principal Investigator at the ELLIS Institute Tübingen. Before this, he spent time at the Universities of Maryland, Siegen and Münster. When it comes to efficient learning, he studies how to build systems that do more with less, from weight averaging techniques to recursive computation approaches that extend model capabilities. with  a particular interest in how these systems reason, and whether we can enhance their reasoning abilities while maintaining efficiency. How do we build mechanisms that let these models learn to be intelligent systems? At the core of his research is also the intersection to safety: Can we make models that reason well without sacrificing safety? How do computational constraints affect safety guarantees? Can we design systems where intelligence and safety reinforce each other?

Abstract: The Promise of Recurrent Depth for Efficient Reasoning

Talk Abstract: Language models with recurrent depth, also referred to as universal or looped when considering transformers, are defined by their capacity to increase their computation through the repetition of layers. Recent pretraining efforts have demonstrated that these architectures can scale to modern language modeling tasks while exhibiting advantages in reasoning tasks. This makes their recurrence in depth an exciting additional axis for scaling model performance, separate from the established ‘verbalized’ chain-of-thought paradigm. In this talk, we’ll discuss recent advances, connect  to classic methods and explore neuroscience motivations.