Integrated Calendar

Lip-to-speech involves generating a natural-sounding speech synchronized with a soundless video of a person talking. Despite recent advances, current methods still cannot produce high-quality speech with high levels of intelligibility for challenging and realistic datasets. This talk presents LipVoicer, a novel method that generates high-quality speech, even for in-the-wild and rich datasets, by incorporating the text modality. Given a silent video, we first predict the spoken text using a pre-trained lip-reading network. We then condition a diffusion model on the video and use the extracted text through a classifier-guidance mechanism where a pre-trained automatic speech recognition (ASR) serves as the classifier. We demonstrate the effectiveness of LipVoicer through human evaluation, which shows that it produces more natural and synchronized speech signals than competing methods (demo page: https://lipvoicer.github.io). The presented LipVoicer is a joint work of Yochai Yemini, Aviv Shamsian, Lior Bracha, Sharon Gannot, and Ethan Fetaya.

Bio:

Sharon Gannot obtained his Ph.D. in electrical engineering from Tel-Aviv University, Israel, in 2000. He is a full professor in the Faculty of Engineering at Bar-Ilan University, Israel. He serves as a senior area chair for IEEE Transactions on Audio, Speech, and Language Processing, a member of the senior editorial board of IEEE Signal Processing Magazine, a member of the editorial board of IEEE SPS Education Center, and the chair of the IEEE Signal Processing Society Data Science Initiative. Previously, he was chair of the IEEE Audio and Acoustic Signal Processing Technical Committee in 2017–2018. He has also held other roles, such as the general co-chair of the 2010 International Workshop on Acoustic Signal Enhancement and the 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. He is the general co-chair of Interspeech 2024, which will be held in September in Greece. His research interests include statistical signal processing and machine learning in the audio processing domain. The methods he develops utilize multi-microphone and multi-modal information. Applications include speech enhancement, noise reduction, speaker separation and diarization, dereverberation, speaker localization, and tracking. Sharon Gannot is the recipient of the 2022 European Association for Signal Processing Group Technical Achievement Award and a Fellow of the IEEE.

 

 

In this talk I will present a useful construction introduced by Kassabov and Pak, called diagonal products. They arise naturally in the study of the space of marked groups, that is, the normal subgroups of a free group with the Chabauty topology. It turns out that diagonal products are extremely rich, and proved to be a useful tool for providing a full spectrum of various growth functions for groups. This includes answers to questions such as "How fast do Følner sets grow in an amenable group" and "How fast do residual chains grow in residually finite groups" etc. We will elaborate on a joint work with Arie Levit and Itamar Vigdorovich, where we give a general classification result for characters (á la Thoma) of many diagonal products. As a result, we deduce that there are uncountably many Hilbert–Schmidt stable groups, which are as unstable as one wants.