Events

"Loop Soup" is a classical statistical physics model, which has a deep connection with many other models, including Gaussian free fields, conformal loop ensembles, loop-erased random walks, uniform spanning trees, the FK-Ising model, (possibly) the phi^4 model, etc. This talk will give an elementary introduction to this model, and present our recent result that the clusters of loop soups on R^3 and the metric graph of Z^3 have different fractal dimensions. This result corrects a key prediction in Wendelin Werner’s blueprint for the scaling limit of metric graph loop soups, and leads to a bunch of open questions. This is a joint work with Jian Ding (Peking University).

Large language models often produce errors—factual inaccuracies, biases, and reasoning failures known as "hallucinations". We show that LLMs' internal representations encode rich information about truthfulness, but in surprising ways. First, truthfulness information concentrates in specific tokens, allowing a dramatic improvement in error detection compared to using other token locations. However, these detectors don't generalize across datasets, revealing that truthfulness encoding is multifaceted rather than universal. Second, internal representations can predict the types of errors a model will make, enabling targeted mitigation strategies. Finally, we uncover a striking discrepancy: LLMs sometimes internally encode correct answers while consistently generating incorrect ones. We'll also discuss follow-up work building on these findings and their implications for developing more reliable language models.

Bio:
Hadas is a Research Fellow at Harvard's Kempner Institute, where she studies the internal mechanics of large AI models to improve their robustness, safety, and reliability. She focuses on problems where scaling compute and data falls short—such as hallucinations, harmful outputs, and biases—with the broader goal of developing controllable AI systems. She completed her PhD in the Technion under the supervision of Yonatan Belinkov.

Modern machine learning systems operate in regimes that challenge classical learning-theoretic assumptions. Models are highly overparameterized, trained with simple optimization algorithms, and rely critically on how data is collected and curated. Understanding the limits of learning in these settings requires revisiting both the computational and statistical foundations of learning theory.

A central question in learning theory asks which functions are tractably learnable. Classical complexity results suggest strong computational barriers, motivating a focus on “learnable subclasses” defined by properties of the target function. In this talk, I argue for a different perspective by emphasizing the role of the training distribution. Fixing the learning algorithm (e.g. stochastic gradient descent applied to neural networks), I show that allowing a “positive distribution shift”, where training data is drawn from a carefully chosen auxiliary distribution while evaluation remains on the target distribution, can render several classically hard learning problems tractable.

Beyond computational considerations, I then study statistical limits of learning in modern, overparameterized models using stochastic convex optimization as a theoretical framework. While classical theory often suggests that successful generalization requires avoiding memorization, I show that memorization is in fact unavoidable: achieving high accuracy requires retaining nontrivial information about the training data and can even enable the identification of individual training examples. These results reveal fundamental privacy–accuracy tradeoffs inherent to accurate learning.

Bio:

Idan Attias is a postdoctoral researcher at the Institute for Data, Econometrics, Algorithms, and Learning (IDEAL), working with Lev Reyzin (University of Illinois Chicago), Nati Srebro, and Avrim Blum (Toyota Technological Institute at Chicago). He obtained his Ph.D. in Computer Science under the supervision of Aryeh Kontorovich (Ben-Gurion University) and Yishay Mansour (Tel Aviv University and Google Research).

His research focuses on the foundations of machine learning theory and data-driven sequential decision-making. His work has been recognized with a Best Paper Award at ICML ’24 and selection as a Rising Star in Data Science (University of California San Diego ’24). His postdoctoral research is supported by an NSF fellowship, and his Ph.D. studies were fully supported by the Israeli Council for Higher Education Scholarship for Outstanding PhD Students in Data Science.

While AI models are becoming an ever-increasing part of our lives, our understanding of their behavior in unexpected situations is drifting even further out of reach. This gap poses significant risks to users, model owners, and society at large.

In the first part of the talk, I will overview my research on detecting unexpected phenomena with and within deep learning models. Specifically, detecting (i) anomalous samples, (ii) unexpected model behavior, and (iii) unexpected security threats. In the second part of the talk, I will dive into my recent research on a specific type of unexpected security threat: attacks on image watermarks. I will review such attacks and present my recent work toward addressing them. I will conclude with a discussion of future research directions.

Bio:

Niv Cohen is a postdoctoral researcher at the school of Computer Science & Engineering at New York University. He received his Ph.D. in Computer Science from the Hebrew University in 2024. His research interests include representation learning, computer vision, and AI safety. He is a recipient of the VATAT Scholarship for Outstanding Postdoctoral Fellows in Data Science and the 2024 Blavatnik Prize for Outstanding Israeli Doctoral Students in Computer Science.

Financial systems are becoming increasingly digital and decentralized, demanding a practical fusion of distributed systems security and economic theory. A key enabler of this change, blockchain technology, promises more private and egalitarian economic mechanisms, built by facilitating consensus between pseudonymous actors. However, the theoretical security of these systems may mask significant real-world risks. In this talk, I will present recent advances in bridging this gap between theory and practice. First, I will discuss the resolution of a decade-old puzzle: the lack of observed attacks on major consensus mechanisms. I will then distill the lessons learnt into a holistic approach to designing robust mechanisms for distributed pseudonymous systems and demonstrate its adoption in practice using several lines of work.