Integrated Calendar

Recent advancements in audio and language processing have yielded significant progress in audio analysis and synthesis. In the realm of audio analysis, researchers are addressing the crucial challenges of Automatic Speech Recognition (ASR), Sound Localization, Event Detection, Emotion Recognition, Speaker Diarization, and Speaker Identification. Meanwhile, in the synthesis domain efforts are focused on Speech Synthesis, Speech Separation, and Audio Vocoders. Despite the progress made, there remains a significant void in the advancement of neural audio generative models that possess the capability to understand audio landscapes and skillfully create or improve new auditory surroundings. In this talk, I will address two pivotal research directions aimed at closing this gap: 

(i) The development of an oracle-powered speechbot involves achieving a profound understanding of the acoustic environment and integrating comprehensive world knowledge. I'll present Spectron, a speechbot that leverages a Large Language Model (LLM) to perform question answering (QA) and speech continuation.

(ii) The second challenge revolves around audio separation for a multitude of sources. While current audio separation literature predominantly focuses on isolating single-source domains like speech or sound events, the real-world scenario demands the separation of diverse sources such as speech, noise, and acoustic events. I will present a solution capable of separating numerous speakers based on a single microphone recording as well as a theoretical upper bound for the single channel speech separation.

Concluding the discussion, I will outline future research directions, focusing on the evolution of multi-agent speechbots, the advancement of generative audio models within the 3D domain, and the fusion of synthetic sounds into real-world environments.

Short Bio:

Eliya Nachmani currently serves as a research scientist at Google Research, specializing in machine learning for audio processing. Prior to his role at Google, he conducted research at Facebook AI Research (FAIR) and pursued his Ph.D. at Tel-Aviv University. Eliya holds a Master of Science in Electrical Engineering from Tel-Aviv University and a Bachelor of Science in Electrical Engineering from the Technion. Website: https://sites.google.com/view/eliya-nachmani/home

Microbial ecosystems, pivotal in global ecological stability, display a diverse array of species, influenced by complex interactions. When considering environments with changing nutrient levels, we have recently suggested an 'early bird' effect. This phenomenon, which results from changing nutrient levels, initial and fast uptake of resources confers an advantage, significantly altering microbial growth dynamics. In serial dilution cultures with varying nutrient levels, this effect leads to shifts in diversity, demonstrating that microbial communities do not adhere to a universal nutrient-diversity relationship. Using a consumer-resource, serial dilution modeling framework, we simulate scenarios of changing nutrient balance, such as variations in phosphorous availability in rainforest soils, to predict a possible lag in ecosystems response near a loss of diversity transition point. Lastly, we explore the notion of 'microbial debt', a form of the early bird advantage, where microbes initially grow rapidly at the cost of later growth or increased mortality. This dynamic, exemplified in both classical chemostat and serial dilution cultures, reveals that such debt can convey an advantage, with varying outcomes on community structure depending on the nature of the trade-off involved. Together, these studies illuminate some organizing principles behind microbial dynamics, balancing growth and survival in changing environments.

Language models (LMs) have vastly improved over the past few years, but they still have not made their way into most people’s daily lives. In this talk, I argue that almost all of the existing language modeling benchmarks are saturated. I explain why I believe that the most important task in NLP right now is to build new, natural and challenging benchmarks that resemble how we want people to use LMs in the real world. I explain why I believe testing language model’s ability to program is the best current method to benchmark them, and discuss a few recent papers in this direction, both mine (SWE-bench) and from other groups.

Mixtures of lipids and cholesterol (Chol) have been served as simple model systems for studying the biophysical principles governing the formation of liquid ordered raft domains in complex biological systems. These mixtures exhibit a rich phase diagram as a function of temperature and composition. Much of the focus in these studies has been given to the coexistence regime between liquid ordered and liquid disordered phases which resembles rafts floating in the sea of disordered lipids. In the talk, I will present a new lattice model of binary [1] and ternary [2, 3] mixtures containing saturated and unsaturated lipids, and Chol. Simulations of mixtures of thousands of lipids and cholesterol molecules on time scales of hundreds of microseconds show a very good agreement with experimental and atomistic simulation observations across multiple scale, ranging from the local distributions of lipids to the macroscopic phase diagram of such mixtures. Importantly, we find that the liquid ordered domains are highly heterogeneous and consist of Chol-poor hexagonally packed gel-like clusters surrounded by Chol-rich regions at the domain boundaries. The presence of such nano-domains within the liquid ordered regions appears as a characteristic feature of the liquid-ordered state, and makes the interpretation of scattering data ambiguous in mixtures not exhibiting macroscopic phase separation.

The talk is based on Nicolas Bergeron’s book, Sections 5.1–5.2.