Integrated Calendar

We propose a new approach for generative modeling based on training a neural network to be idempotent. An idempotent operator is one that can be applied sequentially without changing the result beyond the initial application, namely f(f(z))=f(z). The proposed model f is trained to map a source distribution (e.g, Gaussian noise) to a target distribution (e.g. realistic images) using the following objectives: (1) Instances from the target distribution should map to themselves, namely f(x)=x. We define the target manifold as the set of all instances that f maps to themselves. (2) Instances that form the source distribution should map onto the defined target manifold. This is achieved by optimizing the idempotence term, f(f(z))=f(z) which encourages the range of f(z) to be on the target manifold. Under ideal assumptions such a process provably converges to the target distribution. This strategy results in a model capable of generating an output in one step, maintaining a consistent latent space, while also allowing sequential applications for refinement. Additionally, we find that by processing inputs from both target and source distributions, the model adeptly projects corrupted or modified data back to the target manifold. This work is a first step towards a ``global projector'' that enables projecting any input into a target data distribution. Work done at UC Berkeley

The Origin Recognition Complex (ORC) seeds the replication-fork by binding DNA replication origins, which in budding yeast contain a 17bp DNA motif. High resolution structure of the ORC-DNA complex revealed two base-interacting elements: a disordered basic patch (Orc1-BP4) and an insertion helix (Orc4-IH).
To define ORC elements guiding its DNA binding in-vivo, we mapped genomic locations of 38 designed ORC mutants. We revealed that different ORC elements guide binding at different motifs sites, and these correspond only partially to the structure- described interactions. In particular, we show that disordered basic patches are key for ORC-motif binding in-vivo, including one lacking from the structure. Finally i will discuss how those disordered elements, which insert into the minor-groove can still guide specific ORC-DNA recognition.

Recent advancements in audio and language processing have yielded significant progress in audio analysis and synthesis. In the realm of audio analysis, researchers are addressing the crucial challenges of Automatic Speech Recognition (ASR), Sound Localization, Event Detection, Emotion Recognition, Speaker Diarization, and Speaker Identification. Meanwhile, in the synthesis domain efforts are focused on Speech Synthesis, Speech Separation, and Audio Vocoders. Despite the progress made, there remains a significant void in the advancement of neural audio generative models that possess the capability to understand audio landscapes and skillfully create or improve new auditory surroundings. In this talk, I will address two pivotal research directions aimed at closing this gap: 

(i) The development of an oracle-powered speechbot involves achieving a profound understanding of the acoustic environment and integrating comprehensive world knowledge. I'll present Spectron, a speechbot that leverages a Large Language Model (LLM) to perform question answering (QA) and speech continuation.

(ii) The second challenge revolves around audio separation for a multitude of sources. While current audio separation literature predominantly focuses on isolating single-source domains like speech or sound events, the real-world scenario demands the separation of diverse sources such as speech, noise, and acoustic events. I will present a solution capable of separating numerous speakers based on a single microphone recording as well as a theoretical upper bound for the single channel speech separation.

Concluding the discussion, I will outline future research directions, focusing on the evolution of multi-agent speechbots, the advancement of generative audio models within the 3D domain, and the fusion of synthetic sounds into real-world environments.

Short Bio:

Eliya Nachmani currently serves as a research scientist at Google Research, specializing in machine learning for audio processing. Prior to his role at Google, he conducted research at Facebook AI Research (FAIR) and pursued his Ph.D. at Tel-Aviv University. Eliya holds a Master of Science in Electrical Engineering from Tel-Aviv University and a Bachelor of Science in Electrical Engineering from the Technion. Website: https://sites.google.com/view/eliya-nachmani/home

Microbial ecosystems, pivotal in global ecological stability, display a diverse array of species, influenced by complex interactions. When considering environments with changing nutrient levels, we have recently suggested an 'early bird' effect. This phenomenon, which results from changing nutrient levels, initial and fast uptake of resources confers an advantage, significantly altering microbial growth dynamics. In serial dilution cultures with varying nutrient levels, this effect leads to shifts in diversity, demonstrating that microbial communities do not adhere to a universal nutrient-diversity relationship. Using a consumer-resource, serial dilution modeling framework, we simulate scenarios of changing nutrient balance, such as variations in phosphorous availability in rainforest soils, to predict a possible lag in ecosystems response near a loss of diversity transition point. Lastly, we explore the notion of 'microbial debt', a form of the early bird advantage, where microbes initially grow rapidly at the cost of later growth or increased mortality. This dynamic, exemplified in both classical chemostat and serial dilution cultures, reveals that such debt can convey an advantage, with varying outcomes on community structure depending on the nature of the trade-off involved. Together, these studies illuminate some organizing principles behind microbial dynamics, balancing growth and survival in changing environments.

Language models (LMs) have vastly improved over the past few years, but they still have not made their way into most people’s daily lives. In this talk, I argue that almost all of the existing language modeling benchmarks are saturated. I explain why I believe that the most important task in NLP right now is to build new, natural and challenging benchmarks that resemble how we want people to use LMs in the real world. I explain why I believe testing language model’s ability to program is the best current method to benchmark them, and discuss a few recent papers in this direction, both mine (SWE-bench) and from other groups.