Integrated Calendar

From rough sketches that spark ideas to polished illustrations that explain complex concepts, visual communication is central to how humans think, create, and share knowledge. Yet despite major advances in generative AI, we are still far from models that can reason and communicate through visual forms.

I will present my work on bridging generative models and visual communication, focusing on three complementary domains: (1) algorithms for generating and understanding sketches, (2) systems that support exploratory visual creation beyond one-shot generation, and (3) methods for producing editable, parametric images for design applications.

These domains pose unique challenges: they are inherently data-scarce and rely on representations that go beyond pixel-based images commonly used in standard models. I will show how the rich priors of vision-language models can be leveraged to address these challenges through novel optimization objectives and regularization techniques that connect their learned features with the specialized representations required for visual communication.

Looking ahead, this research lays the foundation for general-purpose visual communication technologies: intelligent systems that collaborate with humans in visual domains, enhancing how we design, learn, and exchange knowledge.

Bio:

Yael Vinker is a Postdoctoral Associate at MIT CSAIL, working with Prof. Antonio Torralba. She received her Ph.D. in Computer Science from Tel Aviv University, advised by Profs. Daniel Cohen-Or and Ariel Shamir. Her research spans computer graphics, computer vision, and machine learning, with a focus on generative models for visual communication. Her work has been recognized with two Best Paper Awards (SIGGRAPH 2022, SIGGRAPH Asia 2023) and a Best Paper Honorable Mention (SIGGRAPH 2023). She was selected as an MIT EECS Rising Star (2024) and received the Blavatnik Prize for Outstanding Israeli Doctoral Students in Computer Science (2024) as well as the VATAT Ph.D. Fellowship.

Satellite instruments, such as TROPOMI, are routinelyused to quantify tropospheric nitrogen dioxide (NO2)based on its narrowband light absorption in the UV/visible spectral range. The key limitation of suchretrievals is that they can only return the „verticalcolumn density“ (VCD), defined as the integral of theNO2 concentration profile. The profile itself, whichdescribes the vertical distribution of NO2, remainsunknown.This presentation showcases „NitroNet“, the first NO2profile retrieval for TROPOMI. NitroNet is a neuralnetwork, which was trained on synthetic NO2 profilesfrom the regional chemistry and transport model WRFChem,operated on a European domain for the month ofMay 2019. The neural network receives NO2 VCDs fromTROPOMI alongside ancillary variables (meteorology,emission data, etc.) as input, from which it estimates NO2concentration profiles.The talk covers:• an introduction to satellite remote sensing of NO2.• the theoretical underpinnings of NitroNet, how themodel was trained, and how it was validated.• practical new applications that NitroNet enables.