לוח שנה משולב

The rapid advancement of Large-scale Text-to-Image models has revolutionized generative tasks by producing high-quality images from textual prompts. However, amid the progress, significant challenges persist, three of which will be the focus of my talk. First, the issue of alignment between the text prompt and the resulting image, particularly evident when multiple subjects are present in the prompt. Second, the challenge of effectively utilizing recent image editing techniques to consistently edit images captured from varying views. Third, the task of distilling the 2D prior knowledge to enhance generation capabilities across diverse domains.

In the first portion I'll give an introduction and motivation for geometric structures on manifolds, and discuss affine structures.  Then I will explain the classification of affine structures on the 2 torus (Nagano and Yagi, Arrowsmith and Furness, Baues, Goldman) and deformations (Baues and Goldman).  Then I will discuss the classification of affine structures on the 3 torus, and progress on the deformation space. Everything is joint work in progress with Max Reistenberg. 

A few years after the deep learning revolution in computer vision it became apparent that despite their amazing performance, DL classifiers are remarkably brittle and can often be fooled by tiny perturbations of the input (e.g. a 1 pixel translation of the image). While these results received much attention, for the most part modern classifiers continue to be trained to optimize accuracy with the hope that robustness will emerge "for free" when training is done on large scale datasets with appropriate data augmentation.

In the first part of my talk, I will present empirical evaluations that show that SOTA image classifiers (e.g. DINOV2) trained on billions of images with data augmentation are still highly brittle. In the second part of the talk, I will present classifiers that we developed using a  more standard, engineering, approach. Rather than hoping that robustness will emerge for free, we try to understand the source of the failure and to create systems that build on the success of DL but are explicitly designed to be  robust. 

Joint work with Hadar Yosef. Amir Hagai and Ofir Shifman.