Integrated Calendar

Foundation models that connect vision and language have recently shown great promise for a wide array of tasks such as text-to-image generation. Significant attention has been devoted towards utilizing the visual representations learned from these powerful vision and language models. In this talk, I will present an ongoing line of research that focuses on the other direction, aiming at understanding what knowledge language models acquire through exposure to images during pretraining. We first consider in-distribution text and demonstrate how multimodally trained text encoders, such as that of CLIP, outperform models trained in a unimodal vacuum, such as BERT, over tasks that require implicit visual reasoning. Expanding to out-of-distribution text, we address a phenomenon known as sound symbolism, which studies non-trivial correlations between particular sounds and meanings across languages and demographic groups, and demonstrate the presence of this phenomenon in vision and language models such as CLIP and Stable Diffusion. Our work provides new angles for understanding what is learned by these vision and language foundation models, offering principled guidelines for designing models for tasks involving visual reasoning.

Bio:
Hadar Averbuch-Elor is an Assistant Professor at the School of Electrical Engineering in Tel Aviv University. Before that, Hadar was a postdoctoral researcher at Cornell-Tech. She completed her PhD in Electrical Engineering at Tel-Aviv University. Hadar is a recipient of several awards including the Zuckerman Postdoctoral Scholar Fellowship, the Schmidt Postdoctoral Award for Women in Mathematical and Computing Sciences, and the Alon Fellowship for the Integration of Outstanding Faculty. She was also selected as a Rising Star in EECS in 2020. Hadar's research interests lie in the intersection of computer graphics and computer vision, particularly in combining pixels with more structured modalities, such as natural language and 3D geometry.

An efficient method to inhibit pathological crystallization is the identification of modifiers, which are (macro)molecules that reduce the rate of crystal growth. Here, I will discuss progress in understanding nonclassical pathways of crystallization and the design of effective modifiers as treatments of three human diseases: kidney stones, malaria, and atherosclerosis. One of the primary tools used to explore crystal growth mechanisms and modifier-crystal interfacial interactions is in situ atomic force microscopy, which we have coupled with microfluidics to assess modifier efficacy. Results from collaborative studies with computational and medical experts have identified unique crystallization pathways, mechanisms of crystal growth inhibition, and promising new therapies, such as the discovery of hydroxycitrate as an inhibitor of calcium oxalate kidney stones. Our studies revealed that hydroxycitrate induces strain in crystals, leading to localized dissolution. A similar outcome was observed for urate stones where solute isomers function as native growth inhibitors that can induce dramatic changes in crystal morphology, and suppress crystal growth at specific conditions. I will discuss new insights into studies of kidney stone prevention and highlight their similarities and differences with novel approaches we have been developing for controlled crystallization in malaria (i.e. heme crystals) and atherosclerosis (i.e. cholesterol crystals).

Succinct non-interactive arguments (SNARGs) are a powerful
cryptographic primitive whose feasibility is still poorly understood.
However, over the last few years, a successful paradigm for building
SNARGs from standard cryptographic assumptions has emerged:
- First, build a non-interactive *batch* argument system (BARG) for NP.
- Then, use BARGs for NP to build SNARGs for various NP languages of
interest.
In this talk, we will discuss recent progress on constructing SNARGs
within this paradigm. Specifically, we study:
1) Under what computational assumptions can we build BARGs for NP?
2) For which NP languages can we build SNARGs within this paradigm?

This talk is based on joint works with Zvika Brakerski, Maya Brodsky,
Yael Kalai, Omer Paneth, Vinod Vaikuntanathan, and Daniel Wichs.

Spatial and temporal regulation of the apoptotic machinery is critical for the execution of multiple cellular events. Here we identify Seven In Absentia Homolog 3 (Siah3) as a new regulator of the cell death machinery during axonal pruning in developing mice. Sensory neurons from Siah3 KO mice exhibit delayed axonal degeneration and Caspase-3 activation in response to trophic deprivation. In agreement, the Siah3 KO mice display increased peripheral sensory innervation. Mechanistically, we show that Siah3 directly binds to the core mitophagy machinery protein Parkin, and, importantly, co-ablation of Prkn and Siah3 reverses the delay in axonal degeneration and Caspase-3 activation detected in Siah3 KO neurons. Strikingly, loss of Siah3 causes dramatic increase in axonal mitophagy upon trophic deprivation, suggesting that Siah3 is a positive regulator of axonal elimination acting by modulation of Parkin-mediated mitophagy. Overall, our results suggest that Parkin-mediated mitophagy restrains the apoptotic system by eliminating signaling mitochondria and reveal the role of mitochondrial signaling in axonal elimination.

When training large neural networks, there are typically many solutions that perfectly fit the training data. Nevertheless, gradient-based methods have a tendency to reach those which generalize well, and understanding this "implicit bias" has been a subject of extensive research. In this talk, I will discuss three works that show settings where the implicit bias provably implies generalization in two-layer neural networks: First, the implicit bias implies generalization in univariate ReLU networks. Second, in ReLU networks where the data consists of clusters and the correlations between cluster means are small, the implicit bias leads to solutions that generalize well, but are highly vulnerable to adversarial examples. Third, in Leaky-ReLU networks (as well as linear classifiers), under certain assumptions on the input distribution, the implicit bias leads to benign overfitting: the estimators interpolate noisy training data and simultaneously generalize well to test data.

Based on joint works with Spencer Frei, Itay Safran, Peter L. Bartlett, Jason D. Lee, and Nati Srebro.


Bio:
Gal is a postdoc at TTI-Chicago and the Hebrew University, hosted by Nati Srebro and Amit Daniely as part of the NSF/Simons Collaboration on the Theoretical Foundations of Deep Learning. Prior to that, he was a postdoc at the Weizmann Institute, hosted by Ohad Shamir, and a PhD student at the Hebrew University, advised by Orna Kupferman. His research focuses on theoretical machine learning, with an emphasis on deep-learning theory.

The intercalation of ions is a powerful strategy to modify the structural, electrical and optical properties of layered solids [1]. It is also a key ingredient for energy storage and the operation of secondary batteries. Even though first studies of driving chemical elements into the van der Waals galleries of graphite date back to as early as 1840, we believe that our recent successful demonstration of on-chip electrochemistry driven ion intercalation in the single van der Waals gallery of a graphene bilayer marks a paradigm shift. The “active” device area is left uncovered by the electrolyte and we can borrow the toolbox of the low dimensional electron system community for monitoring the ion transport [2,3]. This intercalation 2.0 offers, in conjunction with the versatile technique of van der Waals stacking of 2D materials for engineering arbitrary layered structures and hetero-interfaces, unprecedented control and truly unique opportunities to chart new territory in the fields addressing ion transport, diffusion, storage and intercalant induced structural, electronic and optical property changes. Here, examples will be presented how this technique has been exploited to study ion diffusion, ion ordering as well as unconventional superconductivity.
[1] M.S. Dresselhaus, G. Dresselhaus, Intercalation compounds of graphite, Advances in Physics 51-1, 1-186 (2002).
[2] M. Kühne, F. Paolucci, J. Popovic, P. Ostrovsky, J. Maier, J. Smet, Ultrafast lithium diffusion in bilayer graphene, Nature Nanotechnology 12, 895 (2017).
[3] M. Kühne, F. Börrnert, S. Fecher, M. Ghorbani-Asl, J. Biskupek, D. Samuelis, A. Krasheninnikov, U. Kaiser, J. Smet, Reversible superdense ordering of lithium between two graphene sheets, Nature 564, 234-239 (2018).