Sebastian Bordt

Sebastian Bordt

Postdoctoral Researcher in Machine Learning

Hi there! I'm a postdoctoral researcher interested in large language models and interpretability. I work in the theory of machine learning group at the University of Tübingen, supervised by Ulrike von Luxburg.

Currently, I'm very interested in a systematic understanding of pre-training. For a flavor of how I think this can look like, consider this ICML'25 paper. Before that, I did a number of black-box evaluations of LLMs, see here, here, or here. At Microsoft Research, we did some experiments with GPT-4 and GAMs in a healthcare setting.

During my PhD, I worked on a variety of different topics in explainable machine learning. For example, I have worked on the connections between post-hoc methods and interpretable models, and on the suitability of explanation algorithms for regulation. If you are interested in these topics, take a look at this blog post.

Prior to my work in machine learning, I obtained Master's degrees in Mathematics and Economics at TUM and LMU in Munich. I also spent some time at the Munich Graduate School of Economics.

Recent News

May 2025
New preprint about the training dynamics of pre-training, and why standard initialization works surprisingly well: On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling.
October 2024
I gave an invited talk at the Banff Workshop "New Directions in Machine Learning Theory". You can see a recording of the talk here.
July 2024
Our paper on memorization and learning of tabular data in GPT-4 is accepted at COLM.

Selected Publications

For a full list of publications, please see Google Scholar

How Much Can We Forget about Data Contamination?

Sebastian Bordt, Suraj Srinivas, Valentyn Boreiko, Ulrike von Luxburg

ICML 2025

Position: Rethinking Explainable Machine Learning as Applied Statistics

Sebastian Bordt, Eric Raidl, Ulrike von Luxburg

ICML 2025

Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models

Sebastian Bordt, Harsha Nori, Vanessa Rodrigues, Besmira Nushi, Rich Caruana

COLM 2024

Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustness

Suraj Srinivas*, Sebastian Bordt*, Hima Lakkaraju

NeurIPS 2023

Spotlight

From Shapley Values to Generalized Additive Models and back

Sebastian Bordt, Ulrike von Luxburg

AISTATS 2023

Post-hoc explanations fail to achieve their purpose in adversarial contexts

Sebastian Bordt, Michèle Finck, Eric Raidl, Ulrike von Luxburg

FAccT 2022