לוח שנה משולב

NeNeural networks are famously nonlinear. However, linearity is defined relative to a pair of vector spaces, f:X→Y. Is it possible to identify a pair of non-standard vector spaces for which a conventionally nonlinear function is, in fact, linear? This paper introduces a method that makes such vector spaces explicit by construction. We find that if we sandwich a linear operator between two invertible neural networks, then the corresponding vector spaces are induced by newly defined operations. This framework makes the entire arsenal of linear algebra applicable to nonlinear mappings. We demonstrate this by collapsing diffusion model sampling into a single step, enforcing global idempotency for projective generative models, and enabling modular style transfer. 

Bio:

Assaf is an Assistant Professor at the Technion in the Faculty of Data and Decision Sciences. Previously, he was a Research Scientist at NVIDIA, a Postdoc at UC Berkeley with Alyosha Efros, and a Visiting Scholar at Google DeepMind. I received my PhD from the Weizmann Institute of Science, advised by Michal Irani. I have two Bachelor degrees from Ben-Gurion University in Physics and Electrical-Engineering. Assaf’s research focuses on Deep Neural Networks for computer vision, guided by two core principles: a pursuit of elegant, foundational ideas that offer fundamentally new perspectives, and a focus on dynamic and adaptive learning for real-world scenarios like unannotated data streams and distribution shifts.

Real stochastic processes are random real-valued functions on an underlying space (in this talk, Z^d or R^d). Gaussianity occurs when a process is obtained as a sum of many infinitesimal independent contributions, and stationarity occurs when the phenomenon in question is invariant under translations in time or in space. This makes stationary Gaussian processes (SGPs) an excellent model for noise and random signals, placing them amongst the most well studied stochastic processes.
Persistence of a stochastic process is the event of remaining above a fixed level on a large ball of radius T. For a Stationary Gaussian process, we ask two basic questions:
1. What is the asymptotic behavior of the persistence probability, as T grows?
2. Conditioned on the persistence event, what is the typical shape of the process (if there is one)?
These questions, posed by physicists and applied mathematicians decades ago, have been successfully addressed only in the last few years, by exploiting strong relations with harmonic analysis.
In this talk, we will describe old and new results, the main tools and ideas used to achieve them, and many open questions that remain.

Recent years have brought remarkable progress in 3D vision problems like view synthesis and inverse rendering. Despite these advancements, substantial challenges remain in material and lighting decomposition, geometry estimation, and view synthesis—particularly when handling a wide range of materials. In this talk, I will outline a few of these problems and present solutions that combine the principled structure and efficiency of physics-based rendering with the strong priors encoded in generative image and video models.

Bio:

Dor Verbin is a research scientist at Google DeepMind in San Francisco, where he works on computer vision, computer graphics, and machine learning. He received his Ph.D. in computer science from Harvard University. Previously, he received a double B.Sc. in physics and in electrical engineering from Tel Aviv University, after which he worked as a researcher at Camerai, developing real-time computer vision algorithms for mobile devices. He received the Best Student Paper Honorable Mention award at CVPR 2022.

The corners problem is a classical problem in additive combinatorics. A corner is a triple of points (x,y), (x+d,y), (x,y+d). It can be viewed as a 2-dimensional analog of a (one-dimensional) 3-term arithmetic progression. An old question of Ajtai and Szemeredi is: how many points can there be in the n x n integer grid without containing a corner? They proved a qualitative bound of o(n^2), but no effective quantitative bounds.

This question has an equivalent description in the language of communication complexity. Given 3 players with inputs x,y,z which are integers in the range 1 to n, what is the most efficient Number-On-Forehead (NOF) deterministic protocol to check if they sum to n. This connection was first observed in the seminal paper of Chandra, Furst and Lipton that introduced the NOF model back in 1983.

In the language of communication complexity, the trivial protocol sends log(n) bits, but there is a better NOF protocol (based on constructions in additive combinatorics) which only sends (log n)^{1/2} bits. However, the best lower bound until our work was double exponentially far off - of the order of log log log n. In this work, we close this gap, and prove a lower bound of (log n)^c for some absolute constant c.

The work is based on combining the high-level approach of Shkredov, who obtained the previous lower bound, which was based on Fourier analysis; with the recent breakthrough of Kelley and Meka on the 3-term arithmetic progression problem, and the ensuing developments. The main message is that "spreadness" based techniques (a notion that I will explain in the talk) give significantly better quantitative bounds compared to classical Fourier analysis.

Joint work with Michael Jaber, Yang P. Liu, Anthony Ostuni and Mehtaab Sawhney


Paper will appear in FOCS 2025
https://arxiv.org/abs/2504.07006

Despite the remarkable success of modern deep learning, our theoretical understanding remains limited. Many fundamental questions about how these models learn, what they memorize, and what their architectures can express are still largely open. In this talk, I focus on two such questions that offer complementary perspectives on the behavior of modern networks.

First, I examine how standard training procedures implicitly encode aspects of the training data in the learned parameters, enabling reconstruction across a wide range of architectures and loss functions. Second, I turn to transformers and analyze how architectural choices, such as the number of heads, rank, and depth, shape their expressive capabilities, revealing both strengths and inherent limitations of low-rank attention.

Together, these perspectives highlight recurring principles that shape the behavior of deep models, bringing us closer to a theoretical framework that can explain and predict the phenomena observed in practice.

Bio: 

Gilad is a postdoctoral research associate at the Courant Institute of Mathematical Sciences at New York University, hosted by Prof. Joan Bruna. His research focuses on the theory of deep learning models, with a recent emphasis on transformers. He also works on attacks on neural networks, particularly data reconstruction attacks and adversarial attacks. Previously, he completed his Ph.D. at the Weizmann Institute of Science under the supervision of Ohad Shamir. He has held research internships at NVIDIA, where he worked on graph neural networks, and at Google Research, where he worked on large-scale optimization. He holds a B.Sc. and M.Sc. in mathematics from Tel Aviv University.