Projects
Here’s a list of projects where I took the lead and played a central role in driving their development and success.
Dissertation
This work represents the culmination of my research at Heidelberg University, contributing to fundamental and critical questions such as: “How and under which circumstances can we predict reliably under distribution shift?” and “How can we identify failure cases under distribution shift?”
Additionally, the thesis includes a comprehensive introduction to machine learning, domain generalization and causality, providing a solid foundation for the research.
Finding Competence Regions in Domain Generalization
Competence Regions in Domain Generalization (arXiv)
While I initially set out to tackle the problem of making reliable predictions when circumstances change, it became increasingly clear that this task is inherently challenging — and in many cases, impossible. This realization set my focus toward failure detection under distribution shift, addressing the question: Can we detect incorrect predictions when the underlying distribution has shifted, placing everything outside the model’s expected input space?
In this article, I explore the concept of competence regions, areas where a model can be considered reliable, within the context of domain generalization. This approach seeks to better understand where models perform well and where they are likely to fail under distribution shift.
Understanding Context-Aware Domain Generalization
Context Aware Domain Generalization (arXiv)
In my previous work on operationalizing the Principle of Independent Causal Mechanisms (ICM), I explored a method that discards information that varies across distributions to achieve robustness under distribution shift. In this article, I take the opposite approach — aiming to incorporate as much information as possible from the target distribution.
Since labels for a novel target distribution are typically unknown, an effective strategy is to leverage additional samples from the target distribution in the form of a set. Set-encoders, a specific neural architecture designed for handling set-inputs, provide a natural solution for this task. The focus of this article was to establish and evaluate criteria that predict when this approach is likely to succeed, offering valuable insights into the applicability of set-based learning under distribution shift.
Learning Robust Models using the Principle of ICM
When a distribution shifts, some aspects of a system may change, while others remain unaffected. This behavior can be understood through a causal perspective, particularly via the Principle of Independent Causal Mechanisms (ICM), which suggests that different causal components of a system operate independently. Identifying the invariant components holds promise for building more robust models under distribution shift.
In this article, I proposed a method for identifying these components by operationalizing the Principle of ICM. The work includes both a formal mathematical framework (with proofs) and a proof-of-concept experimental analysis demonstrating the approach in practice.
Legalization: A realistic and better alternative?
This article, written during my psychology studies, reflects on the societal harm caused by the war on drugs while exploring the alternative of legalization. It combines a retrospective analysis of past consequences with evidence-based approaches for more effective future policies. (Essay is written in german)