Integrated Calendar

From rough sketches that spark ideas to polished illustrations that explain complex concepts, visual communication is central to how humans think, create, and share knowledge. Yet despite major advances in generative AI, we are still far from models that can reason and communicate through visual forms.

I will present my work on bridging generative models and visual communication, focusing on three complementary domains: (1) algorithms for generating and understanding sketches, (2) systems that support exploratory visual creation beyond one-shot generation, and (3) methods for producing editable, parametric images for design applications.

These domains pose unique challenges: they are inherently data-scarce and rely on representations that go beyond pixel-based images commonly used in standard models. I will show how the rich priors of vision-language models can be leveraged to address these challenges through novel optimization objectives and regularization techniques that connect their learned features with the specialized representations required for visual communication.

Looking ahead, this research lays the foundation for general-purpose visual communication technologies: intelligent systems that collaborate with humans in visual domains, enhancing how we design, learn, and exchange knowledge.

Bio:

Yael Vinker is a Postdoctoral Associate at MIT CSAIL, working with Prof. Antonio Torralba. She received her Ph.D. in Computer Science from Tel Aviv University, advised by Profs. Daniel Cohen-Or and Ariel Shamir. Her research spans computer graphics, computer vision, and machine learning, with a focus on generative models for visual communication. Her work has been recognized with two Best Paper Awards (SIGGRAPH 2022, SIGGRAPH Asia 2023) and a Best Paper Honorable Mention (SIGGRAPH 2023). She was selected as an MIT EECS Rising Star (2024) and received the Blavatnik Prize for Outstanding Israeli Doctoral Students in Computer Science (2024) as well as the VATAT Ph.D. Fellowship.

Likely pathways from the inanimate to the animate world involve a transition that can be thought of as the emergence of information in molecular systems. Before this transition, heteropolymers already exist, but the sequence information of a polymer does not persist beyond the lifetime of the polymer. During the transition, information becomes a separate dynamical entity, which lives and changes over longer timescales than its carrier molecules. This transition does not necessarily require the existence of a full-fledged replication machinery, such as polymerase enzymes that copy informational polymers. Instead, the interplay of simpler reactions can create strong correlations in pools of informational polymers, which can also decouple the timescales of the informational dynamics from those of the polymer dynamics. This interplay is only beginning to be studied, experimentally as well as computationally and theoretically, such that many conceptual and methodological questions are currently open. The aim of my talk is to introduce these questions and present some results, which help to make them more concrete. 

Memory formation and maintenance involve a constant tension between stability andchange. On one hand, stable representations are essential for preserving past experiences.On the other hand, memories must remain flexible to incorporate new information andreflect the evolving world. Thus, while memory resists interference to maintain invariance, itmust also reorganize to enhance efficiency and adapt to novel experiences.In this lecture, I will discuss one to three studies examining this balance within thehippocampal spatial representation system. First, regarding representational drift, spatialrepresentations in the hippocampus gradually change with experience. Our findings suggestthat such changes are driven more by ongoing experience than by forgetting. Second, inexploring environmental mapping, we find that the subiculum encodes differently shapedrooms with strikingly similar activity patterns, hinting at an invariant, latent representation ofspatial structure. Third, we investigate a flashbulb memory–like effect, observingpronounced hippocampal activity changes following salient life events in mice.Together, these projects illustrate how the hippocampus negotiates the trade-off betweenpreserving established memories and accommodating new experiences.

In this talk, we will explore circuit analysis for interpreting neural network models. After some background on the paradigm and techniques of circuit analysis, I'll present two (and a half) research studies demonstrating the breadth of these interpretability methods.

We will explore how this paradigm can help gain scientific insights into how neural network models operate, exemplified in the first work ("Arithmetic without Algorithms", https://technion-cs-nlp.github.io/llm-arithmetic-heuristics) where we use circuit analysis to reveal how language models solve arithmetic prompts. We will also show that circuit analysis can reveal findings on neural network models and help fix existing problems in them --- specifically targeting the issue of poor performance of VLMs on visual tasks compared to equivalent textual tasks (done in the work "Same Task, Different Circuits", https://technion-cs-nlp.github.io/vlm-circuits-analysis). Lastly, if time permits, we will discuss some current directions for future and ongoing work, mainly on scaling circuit analysis to complex tasks.

Bio:

Yaniv Nikankin is a PhD student at the Technion, working with Yonatan Belinkov. His work focuses on interpretability of neural networks, with a recent focus on scaling to analysis of long-form complex tasks. He is particularly excited about cross-domain applications of interpretability in scientific fields, for goals such as better understanding of scientific foundation models such as pLMs. Yaniv is a recipient of the Israeli Higher Education (VATAT) fellowship.

From rough sketches that spark ideas to polished illustrations that explain complex concepts, visual communication is central to how humans think, create, and share knowledge. Yet despite major advances in generative AI, we are still far from models that can reason and communicate through visual forms.

I will present my work on bridging generative models and visual communication, focusing on three complementary domains: (1) algorithms for generating and understanding sketches, (2) systems that support exploratory visual creation beyond one-shot generation, and (3) methods for producing editable, parametric images for design applications.

These domains pose unique challenges: they are inherently data-scarce and rely on representations that go beyond pixel-based images commonly used in standard models. I will show how the rich priors of vision-language models can be leveraged to address these challenges through novel optimization objectives and regularization techniques that connect their learned features with the specialized representations required for visual communication.

Looking ahead, this research lays the foundation for general-purpose visual communication technologies: intelligent systems that collaborate with humans in visual domains, enhancing how we design, learn, and exchange knowledge.

Bio:

Yael Vinker is a Postdoctoral Associate at MIT CSAIL, working with Prof. Antonio Torralba. She received her Ph.D. in Computer Science from Tel Aviv University, advised by Profs. Daniel Cohen-Or and Ariel Shamir. Her research spans computer graphics, computer vision, and machine learning, with a focus on generative models for visual communication. Her work has been recognized with two Best Paper Awards (SIGGRAPH 2022, SIGGRAPH Asia 2023) and a Best Paper Honorable Mention (SIGGRAPH 2023). She was selected as an MIT EECS Rising Star (2024) and received the Blavatnik Prize for Outstanding Israeli Doctoral Students in Computer Science (2024) as well as the VATAT Ph.D. Fellowship.

Prosody, by and large, is the variation in pitch, timing, and loudness that gives speech its musical quality. It is pivotal in human communication, yet its structure and meaning remain subjects of ongoing research. I will describe a data-driven model for English prosody based on large-scale analysis of spontaneous conversations. As a first step, we identified approximately 200 discernible prosodic patterns, i.e., pitch contours typically spanning 1-4 words that we view as building blocks of a prosodic vocabulary, and outlined their properties and communicative meanings. Next, we revealed a Markovian logic, akin to a syntax, affecting how these elementary building blocks concatenate into coherent utterances. We further identified distinct compound functions associated with pairs of consecutive patterns and demonstrated that this Markovian structure is significantly more prevalent in spontaneous prosody compared to scripted speech. These findings offer insights into the underlying mechanisms of conversational prosody, empirically informing and refining existing theoretical concepts in linguistics. The methodology of combining unsupervised clustering analysis of large speech datasets with careful manual annotation could guide future research aimed at refining our model and expanding it to other languages.