Title: Measuring AI Privacy Was Already Hard. Then Came Agents.
Short abstract:
Protecting privacy around AI is hard, and evaluating it is even harder. For a long time, a core concern has been that models leak information about their training data. The first part of this talk illustrates how even that seemingly narrow problem is difficult to evaluate. We focus on heuristic privacy defenses, which trade strong theoretical guarantees (such as differential privacy) for better utility. Without such guarantees, they require particularly careful auditing. Yet we find that typical evaluations can severely underestimate how much models actually leak from memorized training samples.
But AI privacy is not just memorization. Capable AI agents create a broader and less well-defined threat landscape. The second part of this talk explores one slice of this landscape: the misuse of AI agents for privacy attacks. Concretely, we demonstrate that frontier agents can deanonymize pseudonymous online accounts at scale and that they make alarmingly effective tools for stalking. These results suggest that online pseudonymity may no longer be a reliable privacy boundary. More importantly, they illustrate how far the problem has outgrown our tools: we once struggled to measure a threat we understood—now we struggle to even enumerate the threats worth measuring.
Bio:
Michael Aerni is a doctoral candidate in the SPY Lab at ETH Zurich, advised by Prof. Florian Tramèr. He studies the real-world privacy and safety threats of AI systems. His current focus is how capable AI agents can harm the privacy of their users and third parties. Previously, he studied how machine learning models memorize their training data, and how reliably privacy can be measured and defended in practice.
