לוח שנה משולב

The surprising success of learning with deep neural networks poses two fundamental challenges: understanding why these networks work so well and what this success tells us about the nature of intelligence and our biological brain. Our recent Information Theory of Deep Learning shows that large deep networks achieve the optimal tradeoff between training size and accuracy, and that this optimality is achieved through the noise in the learning process.

In this talk, I will focus on the statistical physics aspects of our theory and the interaction between the stochastic dynamics of the training algorithm (Stochastic Gradient Descent) and the phase structure of the Information Bottleneck problem. Specifically, I will describe the connections between the phase transition and the final location and representation of the hidden layers, and the role of these phase transitions in determining the weights of the network.

Based partly on joint works with Ravid Shwartz-Ziv, Noga Zaslavsky, and Shlomi Agmon.

"Sensory disconnection" – conditions when the same sensory stimulus does not reliably affect behavior or subjective experience - is a defining feature of sleep, and similar processes may occur during light anesthesia or during cognitive lapses in wakefulness. What are the changes in brain activity that mediate sensory disconnection? In a series of studies in humans and rodents, we investigate how "disconnected" states affect sensory processing. The first set of studies reveals differences in neuronal responses to identical sensory stimuli across states. We find that in humans, cognitive lapses after sleep deprivation involve attenuated and delayed single-neuron responses in MTL co-occurring with local slow/theta waves. In the auditory domain, we show in both rodents and humans that responses in sleep and light anesthesia are preserved up to A1, challenging the classic "thalamic gating" notion, but robust attenuation occurs later in high-level cortical regions. In addition, sleep affects more strongly responses that require integration over long time intervals, and responses to high-frequency content. The second set of studies investigates the underlying mechanisms, testing the potential role of locus coeruleus-noradrenaline (LC-NE) neuromodulation. In rats, we test how NE signaling affects the probability to wake up from sleep in response to sounds. We establish a new approach for selective in-vivo LC optogenetics by showing effects on spiking activity, evoked sleep-wake transitions, and pupil dilation. Combined LC and auditory stimulation synergistically increases the probability of awakenings beyond independent effects of sound and laser alone, supporting a role for LC-NE activity in mediating sensory responses. We also tested the effects of NE levels on sensory perception and sensory-evoked activity (EEG, fMRI) in awake humans. Pharmacologically manipulating NE levels in double-blind placebo-controlled experiments, we found that NE modulates sensitivity and accuracy of visual perception without significant effects on decision bias (criterion). In addition, NE increased the fidelity of late EEG visual responses, and selectively modulated BOLD fMRI responses in high-order visual cortex, suggesting that NE plays an enabling causal role in visual awareness by affecting late visual processing.

When an ultra-short pulse of radiation is scattered on a complex medium, the emerging radiation pulse is broadened in time. This can be intuitively explained as due to the existence of a large number of paths of varying lengths through which the radiation can traverse the scattering medium. Recently, novel methods to produce ultra-short light pulses were introduced, and they opened a new horizon for experiments were the distribution of delay-times induced by scattering from complex targets can be measured. These developments emphasize the need for theoretical tools to aid planning of new experiments and interpret the measured results. I shall review a general approach for studying the delay-time distribution in both the classical and the quantum (wave) dynamical frameworks. In particular, I shall discuss the delay time distribution in scattering from a random medium where, due to Anderson Localization, there is no classical analogue to this genuine wave phenomenon.

TBA