Integrated Calendar

Lip-to-speech involves generating a natural-sounding speech synchronized with a soundless video of a person talking. Despite recent advances, current methods still cannot produce high-quality speech with high levels of intelligibility for challenging and realistic datasets. This talk presents LipVoicer, a novel method that generates high-quality speech, even for in-the-wild and rich datasets, by incorporating the text modality. Given a silent video, we first predict the spoken text using a pre-trained lip-reading network. We then condition a diffusion model on the video and use the extracted text through a classifier-guidance mechanism where a pre-trained automatic speech recognition (ASR) serves as the classifier. We demonstrate the effectiveness of LipVoicer through human evaluation, which shows that it produces more natural and synchronized speech signals than competing methods (demo page: https://lipvoicer.github.io). The presented LipVoicer is a joint work of Yochai Yemini, Aviv Shamsian, Lior Bracha, Sharon Gannot, and Ethan Fetaya.

Bio:

Sharon Gannot obtained his Ph.D. in electrical engineering from Tel-Aviv University, Israel, in 2000. He is a full professor in the Faculty of Engineering at Bar-Ilan University, Israel. He serves as a senior area chair for IEEE Transactions on Audio, Speech, and Language Processing, a member of the senior editorial board of IEEE Signal Processing Magazine, a member of the editorial board of IEEE SPS Education Center, and the chair of the IEEE Signal Processing Society Data Science Initiative. Previously, he was chair of the IEEE Audio and Acoustic Signal Processing Technical Committee in 2017–2018. He has also held other roles, such as the general co-chair of the 2010 International Workshop on Acoustic Signal Enhancement and the 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. He is the general co-chair of Interspeech 2024, which will be held in September in Greece. His research interests include statistical signal processing and machine learning in the audio processing domain. The methods he develops utilize multi-microphone and multi-modal information. Applications include speech enhancement, noise reduction, speaker separation and diarization, dereverberation, speaker localization, and tracking. Sharon Gannot is the recipient of the 2022 European Association for Signal Processing Group Technical Achievement Award and a Fellow of the IEEE.

 

 

In this talk I will present a useful construction introduced by Kassabov and Pak, called diagonal products. They arise naturally in the study of the space of marked groups, that is, the normal subgroups of a free group with the Chabauty topology. It turns out that diagonal products are extremely rich, and proved to be a useful tool for providing a full spectrum of various growth functions for groups. This includes answers to questions such as "How fast do Følner sets grow in an amenable group" and "How fast do residual chains grow in residually finite groups" etc. We will elaborate on a joint work with Arie Levit and Itamar Vigdorovich, where we give a general classification result for characters (á la Thoma) of many diagonal products. As a result, we deduce that there are uncountably many Hilbert–Schmidt stable groups, which are as unstable as one wants.

The interplay between spontaneous symmetry breaking and topology can result in exotic quantum states of matter. A celebrated example is the quantum anomalous Hall (QAH) effect, which exhibits an integer quantum Hall effect at zero magnetic field due to topologically nontrivial bands and intrinsic magnetism. In the presence of strong electron-electron interactions, fractional-QAH (FQAH) effect at zero magnetic field can emerge, which is a lattice analog of fractional quantum Hall effect without Landau level formation. In this talk, I will present experimental observation of FQAH effect in twisted MoTe2 bilayer, using combined magneto-optical and -transport measurements. In addition, we find an anomalous Hall state near the filling factor -1/2, whose behavior resembles that of the composite Fermi liquid phase in the half-filled lowest Landau level of a two-dimensional electron gas at high magnetic field. Direct observation of the FQAH and associated effects paves the way for researching charge fractionalization and anyonic statistics at zero magnetic field.

Reference
1. Observation of Fractionally Quantized Anomalous Hall Effect, Heonjoon Park et al., Nature, https://www.nature.com/articles/s41586-023-06536-0 (2023);
2. Signatures of Fractional Quantum Anomalous Hall States in Twisted MoTe2 Bilayer, Jiaqi Cai et al., Nature, https://www.nature.com/articles/s41586-023-06289-w (2023);
3. Programming Correlated Magnetic States via Gate Controlled Moiré Geometry, Eric Anderson et al., Science, https://www.science.org/doi/full/10.1126/science.adg4268 (2023).

In this talk, we will prove the topological overlap property for 2-complexes that generalizes the Boros–Furedi theorem, following Yehudayoff's exposition of Gromov's method. The main idea is the notion of a folding map or obstruction, which also has a cohomological interpretation. We will further discuss homological connectivity of random 2-complexes based on a work of Linial and Meshulam.