Integrated Calendar

The mixing time and spectral gap of a random walk on the symmetric group can sometimes be understood in terms of its low dimensional representations (e.g., Aldous' spectral gap conjecture). It turns out that under a mild degree condition involving the step of the group, the same holds for nilpotent groups w.r.t. their one dimensional representations: the spectral gap and the epsilon total variation mixing time of the walk on G are determined by those of the projection of the walk to the abelianization G/[G,G]. We'll discuss some applications concerning the cutoff phenomenon (= abrupt convergence to equilibrium) and the dependence (or lack of!) of the spectral gap and the mixing time on the choice of generators.  

As time permits we shall discuss a related result, confirming in the nilpotent setup a conjecture of Aldous and Diaconis concerning the occurrence of cutoff when a diverging number of generators are picked uniformly at random. Joint work with Zoe Huang.

A defining challenge in synthetic biology is the development of high-throughput and automated methodologies for precise design and engineering of cells. To address these challenges, we develop multiplex genome engineering technologies for versatile genome modification and evolution of bacterial and eukaryotic cells. We use these technologies to create genetic variants to reveal a causal understanding of complex phenotypes as well as engineer pathways and recode genomes. These Genomically Recoded Organisms (GROs) contain alternative genetic codes, in which codons have been eliminated from the genome of E. coli. GROs exhibit improved properties for incorporation of nonstandard amino acids that expand the chemical diversity of proteins or polymers, establish genetic isolation and multi-virus resistance, and enable the engineering of GROs to depend on synthetic amino acids for robust biocontainment strategies. We have also developed new computational-experimental technologies – computer aided design of synthetic genetic elements (CAD-SGE) – that permits the redesign, expression, and mobilization of biosynthetic pathways in diverse organisms for the discovery of new metabolites. This work increases the toolbox for genomic and cellular engineering with broad applications for new classes of enzymes, materials, and therapeutics.

In this talk, I reverse engineer CLIP, one of the most commonly used computer vision backbones. I analyze how individual model components affect the final CLIP representation. I show that the image representation can be decomposed as a sum across individual image patches, model layers, neurons, and attention heads, and use CLIP’s text representation to interpret the summands.

When interpreting the attention heads, each head role can be characterized by automatically finding text representations that span its output space, which reveals property-specific roles for many heads (e.g. location or shape). Next, interpreting the image patches uncovers an emergent spatial localization within CLIP. Finally, the automatic description of the contributions of individual neurons shows polysemantic behavior - each neuron corresponds to multiple, often unrelated, concepts (e.g. ships and cars).

The gained understanding of different components allows three main applications: First, the discovered head roles enable the removal of spurious features from CLIP. Second, emergent localization is used for a strong zero-shot image segmenter. Finally, the extracted neuron polysemy allows the mass production of “semantic” adversarial examples by generating images with concepts spuriously correlated to the incorrect class. The results indicate that a scalable understanding of transformer models is attainable and can be used to detect model bugs, repair them, and improve them.  

BIO:

Yossi is a computer science PhD at UC Berkeley, advised by Alexei Efros, and a visiting researcher at Meta. Before that, he was a member of the perception team at Google Research (now Google-DeepMind). He completed his M.Sc. at Weizmann Institute, advised by Prof. Michal Irani. His research centers around deep learning, computer vision, and mechanistic interpretability.