Integrated Calendar

Text-to-image (T2I) diffusion/flow models achieve state-of-the-art results in image synthesis. Many works leverage these models for real image editing, where a predominant approach involves inverting the image into its corresponding gaussian-like noise map. However, inversion by itself is often insufficient for structure preserving edits. In our first work in this talk, termed ‘An Edit Friendly DDPM Noise Space’ [1], we present alternative latent noise maps for denoising diffusion probabilistic models (DDPMs) that do not have a standard normal distribution. These noise maps allow for perfect reconstruction of any real image, and lead to structure preserving edits, as we exemplify in our experiments.
In our second work, we tackle the task of text-based video editing using T2I diffusion models. Here the main challenge lies in maintaining the temporal consistency of the original video during the edit. Many methods leverage explicit correspondence mechanisms, which struggle with strong nonrigid motion. In contrast, our method termed ‘Slicedit’ [2], introduces a fundamentally different approach, which is based on the observation that spatiotemporal slices of natural videos exhibit similar characteristics to natural images. Thus, the same T2I diffusion model that is normally used only as a prior on video frames, can also serve as a strong prior for enhancing temporal consistency by applying it on spatiotemporal slices. As we show Sliceditgenerates videos that retain the structure and motion of the original video without relying on explicit correspondence matching while adhering to the target text. Finally, in our most recent work, we will discuss ‘FlowEdit’ [3], a novel text-based image editing method that leverages the increasingly popular flow models without relying on inversion. Our method constructs an ODE that directly maps between the source and target distributions (corresponding to the source and target text prompts) and achieves a lower transport cost than the inversion approach. This leads to state-of-the-art results, as we illustrate with Stable Diffusion 3 and FLUX.

[1] An Edit Friendly DDPM Noise Space: Inversion and Manipulations - CVPR24’  https://arxiv.org/abs/2304.06140

[2] Slicedit: Zero-Shot Video Editing With Text-to-Image Diffusion Models Using Spatio-Temporal Slices - ICML24’ https://arxiv.org/abs/2405.12211

[3] FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Models – under review https://arxiv.org/abs/2412.08629

Bio: Vladimir Kulikov, PhD student at the Technion, under the supervision of Prof. Tomer Michaeli. Currently studying Deep Generative Models with emphasis on Computer Vision.

This talk examines how language models can be made more capable through post-training. Although Large Language Models led to major breakthroughs in Natural Language Processing, there remain recurring challenges with Natural Language Understanding. At the core, these challenges are a result of the inherent ambiguity and underspecification of language and this talk will show how they can be addressed with post-training of LMs. The first segment presents the Tulu3 project, exploring language model post-training methodologies. The research proposes open post-training recipes for improving targeted capabilities, with a focus on preference training techniques using Direct Preference Optimization (DPO) and reinforcement learning approaches. The second part investigates preference data through two perspectives. RewardBench, a novel benchmark, systematically evaluates underspecified reward models trained through different methodologies, such as the direct MLE training of classifiers and the implicit reward modeling of DPO. The presentation will detail the benchmark's design, data curation, and comparative performance analysis across varied test sets. Second, the talk will discuss when and how preference annotations diverge, i.e. lead to disagreements between annotators, showing that a lot of disagreements stem from underspecification. It will further explain how the standard Bradley-Terry model does not appropriately capture diverging preferences and will instead suggest different methods to model and detect potentially diverging instances in the data. The final section introduces ClarifyDelphi, a reinforcement learning system for generating clarification questions. Using a defeasibility reward mechanism, the system aims to extract contextual nuances from underspecified social and moral scenarios, demonstrating an approach to more sophisticated contextual reasoning.The talk synthesizes these research threads to illustrate strategies for adapting language model capabilities and improving contextual understanding, when encountering underspecification.

In the last decade there have been several efforts to use physical systems as 'physical computers': D-Wave is using super-conducting qubits as an adiabatic quantum (or non-quantum?) computer, the Israeli company Light-Solver (ex Davidson group) tries to use lasers in finding optimal solutions to optimization problems, HP is developing memristor computation, 'Natural Computing' is developing a chip that is based on thermal computin,  companies like Toshiba and Fujitsu are commercializing products like Bifurcation machines' and Digital annealers', and so on...  Are we indeed seeing the `comeback' of analog computers? In the talk I will discuss the physical ideas behind these machines and try to provide some intuition and understanding on their advantages and disadvantages. FOR THE LATEST UPDATES AND CONTENT ON SOFT MATTER AND BIOLOGICAL PHYSICS AT THE WEIZMANN, VISIT OUR WEBSITE: https://www.biosoftweizmann.com/

Humans adapt their behavior across multiple timescales: from rapid adjustments to changing contexts to lifelong tendencies in how they approach tasks. This variation across time and individuals poses a challenge for identifying the cognitive strategies people use and the neural processes that support them. My research combines computational modeling and neuroimaging to uncover the strategies individuals use and reveal how their dynamics are reflected in neural activity and constrained by brain structure. In this talk I will present my work on computational modeling of cognitive dynamics over weeks. I will briefly describe my work on mapping of human white matter, and my current work on the computational and neural bases of creative search. I will conclude by outlining my future research aimed at uncovering the core principles that drive both the dynamics and diversity of human cognition.

Given a dynamical system which admits chaos, its possible equilibria are studied through the collection of its invariant measures (i.e the probability of an event does not change under time-evolution). This collection may be very large, and we often wish to single out measures of importance. In particular, physical measures are a sought-after object, as they describe an observable equilibrium. In this talk we will define what are physical measures (in a broad sense, including SRB measure), and list a few recent results and open questions regarding their existence for smooth dynamical systems. In particular, finding “testable” conditions for the existence of physical measures is an on-going and active field of research.

Generative models, such as ChatGPT and DALL-E, are used by millions of people daily for tasks ranging from programming and content creation to resume filtering. These models often create the impression of being “intelligent,” which can incentivize careless use in critical applications. While generative models are empowering, they appear to be black boxes, and their misuse can result in harmful or unlawful outcomes.

In this talk, I will present algorithms and tools for dissecting and analyzing generative models using holistic, causal, and data-centric approaches.

By applying these methods to state-of-the-art models, we can foster trust in these technologies by uncovering human-interpretable concepts that underpin their behavior, scrutinizing their extensive training data, and evaluating their learning processes.

Finally, I will reflect on how generative models have transformed the field of AI and discuss the challenges that remain in ensuring their responsible development and use.