Sebastian Bordt

Sebastian Bordt

Postdoctoral Researcher in Machine Learning

Hi there! I'm a postdoctoral researcher interested in large language models and interpretability. I work in the theory of machine learning group at the University of Tübingen with Ulrike von Luxburg.

Currently, I'm working on a systematic understanding of pre-training. For a flavor of how this can look like, consider this preprint or this ICML'25 paper. Before that, I did a number of black-box evaluations of LLMs, see here, here, or here. At Microsoft Research, we did some experiments with GPT-4 and GAMs in a healthcare setting.

During my PhD, I worked on a variety of different topics in explainable machine learning. For example, I have worked on the connections between post-hoc methods and interpretable models, and on the suitability of explanation algorithms for regulation. If you are interested in these topics, take a look at this blog post.

Prior to my work in machine learning, I obtained Master's degrees in Mathematics and Economics at TUM and LMU in Munich. I also spent some time at the Munich Graduate School of Economics.

Recent News

September 2025
New preprint: Train Once, Answer All: Many Pretraining Experiments for the Cost of One. We propose a novel approach how academia can contribute to pretraining research!
September 2025
October 2024
I gave an invited talk at the Banff Workshop "New Directions in Machine Learning Theory". You can see a recording of the talk here.

Selected Publications

For a full list of publications, please see Google Scholar

On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling

Moritz Haas, Sebastian Bordt, Ulrike von Luxburg, Leena Chennuru Vankadara

NeurIPS 2025

Spotlight

How Much Can We Forget about Data Contamination?

Sebastian Bordt, Suraj Srinivas, Valentyn Boreiko, Ulrike von Luxburg

ICML 2025

Position: Rethinking Explainable Machine Learning as Applied Statistics

Sebastian Bordt, Eric Raidl, Ulrike von Luxburg

ICML 2025

Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models

Sebastian Bordt, Harsha Nori, Vanessa Rodrigues, Besmira Nushi, Rich Caruana

COLM 2024

Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustness

Suraj Srinivas*, Sebastian Bordt*, Hima Lakkaraju

NeurIPS 2023

Spotlight

From Shapley Values to Generalized Additive Models and back

Sebastian Bordt, Ulrike von Luxburg

AISTATS 2023