In this talk, I present an approach for interpreting the internal computation in deep vision models. I show that these interpretations can be used to detect model bugs and to improve the performance of pre-trained deep neural networks (e.g., reducing hallucinations from image captioners and detecting and removing spurious correlations in CLIP) without any additional training. Moreover, the obtained understanding of deep representations can unlock new model capabilities (e.g., novel identity editing techniques in diffusion models and faithful image inversion in GANs). I demonstrate how to find common representations across different models (discriminative and generative) and how deep representations can be adapted at test-time to improve model generalization without any additional supervision. Finally, I discuss future work on improving the presented interpretation techniques and their application to continual model correction and scientific discovery.
Bio: Yossi is a EECS PhD at UC Berkeley, advised by Alexei Efros, and a visiting researcher at Meta. Before that, he was a member of the perception team at Google Research (now Google-DeepMind). He completed his M.Sc. at Weizmann Institute, advised by Prof. Michal Irani. His research centers around deep learning, computer vision, and mechanistic interpretability.
