Integrated Calendar

One of the puzzling phenomena in deep learning, is that neural networks tend to generalize well even when they are highly overparameterized. Recent works linked this behavior with implicit biases of the algorithms used to train networks (like SGD). Here we analyze the implicit bias of SGD from the standpoint of minima stability, focusing on shallow ReLU networks trained with a quadratic loss. Specifically, it is known that SGD can stably converge only to minima that are flat enough w.r.t. its step size. Here we show that this property enforces the predictor function to become smoother as the step size increases, thus significantly regularizing the solution. Furthermore, we analyze the representation power of stable solutions. Particularly, we prove a depth-separation result: There exist functions that cannot be approximated by depth-2 networks corresponding to stable minima, no matter how small the step size is taken to be, but which can be implemented with depth-3 networks corresponding to stable minima. We show how our theoretical findings explain behaviors observed in practical settings.
(Joint works with Rotem Mulayoff, Mor Shpigel Nacson, Greg Ongie, Daniel Soudry).

Understanding the neural bases of behavior and cognition requires determining how mechanistically distinct processing elements combine to carry out brain function at an integrated level. In this talk, I will introduce some of our laboratory’s efforts to address this goal using a combination of molecular sensors with noninvasive wide-field imaging. In the first part of the talk, I will discuss how workhorse optical neuroimaging approaches have inspired the design of molecular MRI probes for sensing physiological variables. Some of these probes detect light, providing a means for deep-tissue MRI-assisted optical imaging. I will next introduce an alternative molecular imaging concept inspired by widely used hemodynamic functional MRI techniques. By reengineering some of the proteins and peptides involved in neurovascular coupling, it is possible to create sensitive probes for a variety of neurobiological targets. I will illustrate how this strategy can be used to elucidate patterns of information flow and neurochemically specific functional connectivity in brain circuitry, with anticipated utility for deciphering mechanisms of learning and sensory processing in rodents and primates.

How can we convey a complex idea - a technology, theory, product or initiative - in a clear, fascinating and effective way?
This is a question that thousands of professors, teachers and engineers wrestle with every time they need to present or teach a complex idea. Fortunately, the same proven techniques that have been used by generations of writers, playwrights and directors - come to our aid even in the twenty-first century.
In the lecture we will talk about:
- How to get inside the view's head,
- How to identify vital or redundant pieces of information,
- How to strengthen our presentation by using ideas that contradict and even oppose our ideas! and much more.