Publications
-
(2022) Phys Rev X. 12, p. 021051 Abstract
The mapping of the wiring diagrams of neural circuits promises to allow us to link the structure and function of neural networks. Current approaches to analyzing such connectomes rely mainly on graph-theoretical tools, but these may downplay the complex nonlinear dynamics of single neurons and the way networks respond to their inputs. Here, we measure the functional similarity of simulated networks of neurons, by quantifying the similitude of their spiking patterns in response to the same stimuli. We find that common graph-theory metrics convey little information about the similarity of networks’ responses. Instead, we learn a functional metric between networks based on their synaptic differences and show that it accurately predicts the similarity of novel networks, for a wide range of stimuli. We then show that a sparse set of architectural features—the sum of synaptic inputs that each neuron receives and the sum of each neuron’s synaptic outputs—predicts the functional similarity of networks of up to 1000 neurons, with high accuracy. We thus suggest new architectural design principles that shape the function of neural networks. These architectural features conform with experimental evidence of homeostatic synaptic mechanisms.
-
(2021) Neuron. 109, p. 1-13 Abstract
Learning new rules and adopting novel behavioral policies is a prominent adaptive behavior of primates. Westudied the dynamics of single neurons in the dorsal anterior cingulate cortex and putamen of monkeys whilethey learned new classification tasks every few days over a fixed set of multi-cue patterns. Representing therules and the neuronal selectivity as vectors in the space spanned by a set of stimulus features allowed us tocharacterize neuronal dynamics in geometrical terms. We found that neurons in the cingulate cortex mainlyrotated toward the rule, implying a policy search, whereas neurons in the putamen showed a magnitude in-crease that followed the rotation of cortical neurons, implying strengthening of confidence for the newly ac-quired rule-based policy. Further, the neural representation at the end of a session predicted next-daybehavior, reflecting overnight retention. The novel framework for characterization of neural dynamics sug-gests complementing roles for the putamen and the anterior cingulate cortex.
-
(2021) Advances in Neural Information Processing Systems (NeurIPS 2021). 35, Abstract
The integration and transfer of information from multiple sources to multiple tar-
gets is a core motive of neural systems. The emerging field of partial information
decomposition (PID) provides a novel information-theoretic lens into these mecha-
nisms by identifying synergistic, redundant, and unique contributions to the mutual
information between one and several variables. While many works have studied
aspects of PID for Gaussian and discrete distributions, the case of general contin-
uous distributions is still uncharted territory. In this work we present a method
for estimating the unique information in continuous distributions, for the case of
one versus two variables. Our method solves the associated optimization problem
over the space of distributions with fixed bivariate marginals by combining copula
decompositions and techniques developed to optimize variational autoencoders.
We obtain excellent agreement with known analytic results for Gaussians, and illus-
trate the power of our new approach in several brain-inspired neural models. Our
method is capable of recovering the effective connectivity of a chaotic network of
rate neurons, and uncovers a complex trade-off between redundancy, synergy and
unique information in recurrent networks trained to solve a generalized XOR task. -
(2021) bioRxiv 2021.12.14.472335. Abstract[All authors]
How sexually dimorphic behavior is encoded in the nervous system is poorly understood. Here, we characterize the dimorphic nociceptive behavior in C. elegans and study the underlying circuits, which are composed of the same neurons but are wired differently. We show that while sensory transduction is similar in the two sexes, the downstream network topology markedly shapes behavior. We fit a network model that replicates the observed dimorphic behavior in response to external stimuli, and use it to predict simple network rewirings that would switch the behavior between the sexes. We then show experimentally that these subtle synaptic rewirings indeed flip behavior. Strikingly, when presented with aversive cues, rewired males were compromised in finding mating partners, suggesting that network topologies that enable efficient avoidance of noxious cues have a reproductive “cost”. Our results present a deconstruction of the design of a neural circuit that controls sexual behavior, and how to reprogram it.
-
(2020) Proceedings of the National Academy of Sciences . 117 , 40, p. 25066-25073 Abstract
We present a theory of neural circuits’ design and function, inspired by the random connectivity of real neural circuits and the mathematical power of random projections. Specifically, we introduce a family of statistical models for large neural population codes, a straightforward neural circuit architecture that would implement these models, and a biologically plausible learning rule for such circuits. The resulting neural architecture suggests a design principle for neural circuit—namely, that they learn to compute the mathematical surprise of their inputs, given past inputs, without an explicit teaching signal. We applied these models to recordings from large neural populations in monkeys’ visual and prefrontal cortices and show them to be highly accurate, efficient, and scalable.
-
(2020) eLife. 9, p. e56196 Abstract
The social interactions underlying group foraging and their benefits have been mostly studied using mechanistic models replicating qualitative features of group behavior, and focused on a single resource or a few clustered ones. Here, we tracked groups of freely foraging adult zebrafish with spatially dispersed food items and found that fish perform stereotypical maneuvers when consuming food, which attract neighboring fish. We then present a mathematical model, based on inferred functional interactions between fish, which accurately describes individual and group foraging of real fish. We show that these interactions allow fish to combine individual and social information to achieve near-optimal foraging efficiency and promote income equality within groups. We further show that the interactions that would maximize efficiency in these social foraging models depend on group size, but not on food distribution - suggesting that fish may adaptively pick the subgroup of neighbors they 'listen to' to determine their own behavior.
-
(2020) arxiv. 2006.11671.
-
(2019) Nature Neuroscience. 22, 12, p. 2013-2022 Abstract
The prefrontal cortex (PFC) plays an important role in regulating social functions in mammals, and its dysfunction has been linked to social deficits in neurodevelopmental disorders. Yet little is known of how the PFC encodes social information and how social representations may be altered in such disorders. Here, we show that neurons in the medial PFC of freely behaving male mice preferentially respond to socially relevant olfactory cues. Population activity patterns in this region differed between social and nonsocial stimuli and underwent experience-dependent refinement. In mice lacking the autism-associated gene Cntnap2, both the categorization of sensory stimuli and the refinement of social representations were impaired. Noise levels in spontaneous population activity were higher in Cntnap2 knockouts and correlated with the degree to which social representations were disrupted. Our findings elucidate the encoding of social sensory cues in the medial PFC and provide a link between altered prefrontal dynamics and autism-associated social dysfunction.
-
(2019) bioRxiv. 693192. Abstract
We studied the fine temporal structure of spiking patterns of groups of up to 100 simultaneously recorded units in the prefrontal cortex of monkeys performing a visual discrimination task. We characterized the vocabulary of population activity patterns using 10 ms time bins and found that different sets of population activity patterns (codebooks) are used in different task epochs and that spiking correlations between units play a large role in defining those codebooks. Models that ignore those correlations fail to capture the population codebooks in all task epochs. Further, we show that temporal sequences of population activity patterns have strong history-dependence and are governed by different transition probabilities between patterns and different correlation time scales, in the different task epochs, suggesting different computational dynamics governing each epoch. Together, the large impact of spatial and temporal correlations on the dynamics of the population code makes the observed sequences of activity patterns many orders of magnitude more likely to appear than predicted by models that ignore these correlations and rely only on the population rates. Surprisingly, however, models that ignore these correlations perform quite well for decoding behavior from population responses. The difference of encoding and decoding complexity of the neural codebook suggests that one of the goals of the complex encoding scheme in the prefrontal cortex is to accommodate simple decoders that do not have to learn correlations.
-
(2018) PLoS One. 13, 3, 0193049. Abstract
Recent developments in automated tracking allow uninterrupted, high-resolution recording of animal trajectories, sometimes coupled with the identification of stereotyped changes of body pose or other behaviors of interest. Analysis and interpretation of such data represents a challenge: the timing of animal behaviors may be stochastic and modulated by kinematic variables, by the interaction with the environment or with the conspecifics within the animal group, and dependent on internal cognitive or behavioral state of the individual. Existing models for collective motion typically fail to incorporate the discrete, stochastic, and internal-state- dependent aspects of behavior, while models focusing on individual animal behavior typically ignore the spatial aspects of the problem. Here we propose a probabilistic modeling framework to address this gap. Each animal can switch stochastically between different behavioral states, with each state resulting in a possibly different law of motion through space. Switching rates for behavioral transitions can depend in a very general way, which we seek to identify from data, on the effects of the environment as well as the interaction between the animals. We represent the switching dynamics as a Generalized Linear Model and show that: (i) forward simulation of multiple interacting animals is possible using a variant of the Gillespie's Stochastic Simulation Algorithm; (ii) formulated properly, the maximum likelihood inference of switching rate functions is tractably solvable by gradient descent; (iii) model selection can be used to identify factors that modulate behavioral state switching and to appropriately adjust model complexity to data. To illustrate our framework, we apply it to two synthetic models of animal motion and to real zebrafish tracking data.
-
(2017) Proceedings of the National Academy of Sciences of the United States of America. 114, 38, p. 10149-10154 Abstract
Individual computations and social interactions underlying collective behavior in groups of animals are of great ethological, behavioral, and theoretical interest. While complex individual behaviors have successfully been parsed into small dictionaries of stereotyped behavioral modes, studies of collective behavior largely ignored these findings; instead, their focus was on inferring single, mode-independent social interaction rules that reproduced macroscopic and often qualitative features of group behavior. Here, we bring these two approaches together to predict individual swimming patterns of adult zebrafish in a group. We show that fish alternate between an "active" mode, in which they are sensitive to the swimming patterns of conspecifics, and a "passive" mode, where they ignore them. Using a model that accounts for these two modes explicitly, we predict behaviors of individual fish with high accuracy, outperforming previous approaches that assumed a single continuous computation by individuals and simple metric or topological weighing of neighbors' behavior. At the group level, switching between active and passive modes is uncorrelated among fish, but correlated directional swimming behavior still emerges. Our quantitative approach for studying complex, multi-modal individual behavior jointly with emergent group behavior is readily extensible to additional behavioral modes and their neural correlates as well as to other species.
-
(2017) Proceedings of the National Academy of Sciences of the United States of America. 114, 22, p. 5589-5594 Abstract
Individual behavior, in biology, economics, and computer science, is often described in terms of balancing exploration and exploitation. Foraging has been a canonical setting for studying reward seeking and information gathering, from bacteria to humans, mostly focusing on individual behavior. Inspired by the gradient-climbing nature of chemotaxis, the infotaxis algorithm showed that locally maximizing the expected information gain leads to efficient and ethological individual foraging. In nature, as well as in theoretical settings, conspecifics can be a valuable source of information about the environment. Whereas the nature and role of interactions between animals have been studied extensively, the design principles of information processing in such groups are mostly unknown. We present an algorithm for group foraging, which we term "socialtaxis," that unifies infotaxis and social interactions, where each individual in the group simultaneously maximizes its own sensory information and a social information term. Surprisingly, we show that when individuals aim to increase their information diversity, efficient collective behavior emerges in groups of opportunistic agents, which is comparable to the optimal group behavior. Importantly, we show the high efficiency of biologically plausible socialtaxis settings, where agents share little or no information and rely on simple computations to infer information from the behavior of their conspecifics. Moreover, socialtaxis does not require parameter tuning and is highly robust to sensory and behavioral noise. We use socialtaxis to predict distinct optimal couplings in groups of selfish vs. altruistic agents, reflecting how it can be naturally extended to study social dynamics and collective computation in general settings.
-
(2016) Nature Neuroscience. 19, 11, p. 1489-1496 Abstract[All authors]
Social encounters are associated with varying degrees of emotional arousal and stress. The mechanisms underlying adequate socioemotional balance are unknown. The medial amygdala (MeA) is a brain region associated with social behavior in mice. Corticotropin-releasing factor receptor type-2 (CRF-R2) and its specific ligand urocortin-3 (Ucn3), known components of the behavioral stress response system, are highly expressed in the MeA. Here we show that mice deficient in CRF-R2 or Ucn3 exhibit abnormally low preference for novel conspecifics. MeA-specific knockdown of Crfr2 (Crhr2) in adulthood recapitulated this phenotype. In contrast, pharmacological activation of MeA CRF-R2 or optogenetic activation of MeA Ucn3 neurons increased preference for novel mice. Furthermore, chemogenetic inhibition of MeA Ucn3 neurons elicited pro-social behavior in freely behaving groups of mice without affecting their hierarchal structure. These findings collectively suggest that the MeA Ucn3 CRF-R2 system modulates the ability of mice to cope with social challenges.
-
(2016) Current Opinion in Neurobiology. 37, p. 133-140 Abstract
The ability to record the joint activity of large groups of neurons would allow for direct study of information representation and computation at the level of whole circuits in the brain. The combinatorial space of potential population activity patterns and neural noise imply that-it would be impossible to directly map the relations between stimuli and population responses. Understanding of large neural population codes therefore depends on identifying simplifying design principles. We review recent results showing that strongly correlated population codes can be explained using minimal models that rely on low order relations among cells. We discuss the implications for large populations, and how such models allow for mapping the semantic organization of the neural codebook and stimulus space, and decoding.
-
(2015) eLife. 4, e06134. Abstract
Information is carried in the brain by the joint spiking patterns of large groups of noisy, unreliable neurons. This noise limits the capacity of the neural code and determines how information can be transmitted and read-out. To accurately decode, the brain must overcome this noise and identify which patterns are semantically similar. We use models of network encoding noise to learn a thesaurus for populations of neurons in the vertebrate retina responding to artificial and natural videos, measuring the similarity between population responses to visual stimuli based on the information they carry. This thesaurus reveals that the code is organized in clusters of synonymous activity patterns that are similar in meaning but may differ considerably in their structure. This organization is highly reminiscent of the design of engineered codes. We suggest that the brain may use this structure and show how it allows accurate decoding of novel stimuli from novel spiking patterns.
-
(2014) PLoS Computational Biology. 10, 1, e1003408. Abstract
Maximum entropy models are the least structured probability distributions that exactly reproduce a chosen set of statistics measured in an interacting network. Here we use this principle to construct probabilistic models which describe the correlated spiking activity of populations of up to 120 neurons in the salamander retina as it responds to natural movies. Already in groups as small as 10 neurons, interactions between spikes can no longer be regarded as small perturbations in an otherwise independent system; for 40 or more neurons pairwise interactions need to be supplemented by a global interaction that controls the distribution of synchrony in the population. Here we show that such "K-pairwise'' models-being systematic extensions of the previously used pairwise Ising models-provide an excellent account of the data. We explore the properties of the neural vocabulary by: 1) estimating its entropy, which constrains the population's capacity to represent visual information; 2) classifying activity patterns into a small set of metastable collective modes; 3) showing that the neural codeword ensembles are extremely inhomogenous; 4) demonstrating that the state of individual neurons is highly predictable from the rest of the population, allowing the capacity for error correction.
-
(2014) PLoS One. 9, 1, Abstract
Adaptation in the retina is thought to optimize the encoding of natural light signals into sequences of spikes sent to the brain. While adaptive changes in retinal processing to the variations of the mean luminance level and second-order stimulus statistics have been documented before, no such measurements have been performed when higher-order moments of the light distribution change. We therefore measured the ganglion cell responses in the tiger salamander retina to controlled changes in the second (contrast), third (skew) and fourth (kurtosis) moments of the light intensity distribution of spatially uniform temporally independent stimuli. The skew and kurtosis of the stimuli were chosen to cover the range observed in natural scenes. We quantified adaptation in ganglion cells by studying linear-nonlinear models that capture well the retinal encoding properties across all stimuli. We found that the encoding properties of retinal ganglion cells change only marginally when higher-order statistics change, compared to the changes observed in response to the variation in contrast. By analyzing optimal coding in LN-type models, we showed that neurons can maintain a high information rate without large dynamic adaptation to changes in skew or kurtosis. This is because, for uncorrelated stimuli, spatio-temporal summation within the receptive field averages away non-gaussian aspects of the light intensity distribution.
-
(2013) PLoS One. 8, 10, Abstract
The visual system continually adjusts its sensitivity to the statistical properties of the environment through an adaptation process that starts in the retina. Colour perception and processing is commonly thought to occur mainly in high visual areas, and indeed most evidence for chromatic colour contrast adaptation comes from cortical studies. We show that colour contrast adaptation starts in the retina where ganglion cells adjust their responses to the spectral properties of the environment. We demonstrate that the ganglion cells match their responses to red-blue stimulus combinations according to the relative contrast of each of the input channels by rotating their functional response properties in colour space. Using measurements of the chromatic statistics of natural environments, we show that the retina balances inputs from the two ( red and blue) stimulated colour channels, as would be expected from theoretical optimal behaviour. Our results suggest that colour is encoded in the retina based on the efficient processing of spectral information that matches spectral combinations in natural scenes on the colour processing level.
-
(2013) eLife. 2, Abstract
Social behavior in mammals is often studied in pairs under artificial conditions, yet groups may rely on more complicated social structures. Here, we use a novel system for tracking multiple animals in a rich environment to characterize the nature of group behavior and interactions, and show strongly correlated group behavior in mice. We have found that the minimal models that rely only on individual traits and pairwise correlations between animals are not enough to capture group behavior, but that models that include third-order interactions give a very accurate description of the group. These models allow us to infer social interaction maps for individual groups. Using this approach, we show that environmental complexity during adolescence affects the collective group behavior of adult mice, in particular altering the role of high-order structure. Our results provide new experimental and mathematical frameworks for studying group behavior and social interactions.
-
(2013) PLoS Computational Biology. 9, 3, Abstract
Neural populations encode information about their stimulus in a collective fashion, by joint activity patterns of spiking and silence. A full account of this mapping from stimulus to neural activity is given by the conditional probability distribution over neural codewords given the sensory input. For large populations, direct sampling of these distributions is impossible, and so we must rely on constructing appropriate models. We show here that in a population of 100 retinal ganglion cells in the salamander retina responding to temporal white-noise stimuli, dependencies between cells play an important encoding role. We introduce the stimulus-dependent maximum entropy (SDME) model-a minimal extension of the canonical linear-nonlinear model of a single neuron, to a pairwise-coupled neural population. We find that the SDME model gives a more accurate account of single cell responses and in particular significantly outperforms uncoupled models in reproducing the distributions of population codewords emitted in response to a stimulus. We show how the SDME model, in conjunction with static maximum entropy models of population vocabulary, can be used to estimate information-theoretic quantities like average surprise and information transmission in a neural population.
-
(2013) Proceedings of the National Academy of Sciences of the United States of America. 110, 2, p. 684-689 Abstract
Pattern classification learning tasks are commonly used to explore learning strategies in human subjects. The universal and individual traits of learning such tasks reflect our cognitive abilities and have been of interest both psychophysically and clinically. From a computational perspective, these tasks are hard, because the number of patterns and rules one could consider even in simple cases is exponentially large. Thus, when we learn to classify we must use simplifying assumptions and generalize. Studies of human behavior in probabilistic learning tasks have focused on rules in which pattern cues are independent, and also described individual behavior in terms of simple, single-cue, feature-based models. Here, we conducted psychophysical experiments in which people learned to classify binary sequences according to deterministic rules of different complexity, including high-order, multicue-dependent rules. We show that human performance on such tasks is very diverse, but that a class of reinforcement learning-like models that use a mixture of features captures individual learning behavior surprisingly well. These models reflect the important role of subjects' priors, and their reliance on high-order features even when learning a low-order rule. Further, we show that these models predict future individual answers to a high degree of accuracy. We then use these models to build personally optimized teaching sessions and boost learning.
-
(2013) Physical Review Letters. 110, 5, 058104. Abstract
The ability of an organism to distinguish between various stimuli is limited by the structure and noise in the population code of its sensory neurons. Here we infer a distance measure on the stimulus space directly from the recorded activity of 100 neurons in the salamander retina. In contrast to previously used measures of stimulus similarity, this "neural metric" tells us how distinguishable a pair of stimulus clips is to the retina, based on the similarity between the induced distributions of population responses. We show that the retinal distance strongly deviates from Euclidean, or any static metric, yet has a simple structure: we identify the stimulus features that the neural population is jointly sensitive to, and show the support-vector-machine-like kernel function relating the stimulus and neural response spaces. We show that the non-Euclidean nature of the retinal distance has important consequences for neural decoding.
-
(2012) Proceedings of the National Academy of Sciences of the United States of America. 109, 49, p. 19959-19964 Abstract
In vision, two mixtures, each containing an independent set of many different wavelengths, may produce a common color percept termed "white." In audition, two mixtures, each containing an independent set of many different frequencies, may produce a common perceptual hum termed "white noise." Visual and auditory whites emerge upon two conditions: when the mixture components span stimulus space, and when they are of equal intensity. We hypothesized that if we apply these same conditions to odorant mixtures, "whiteness" may emerge in olfaction as well. We selected 86 molecules that span olfactory stimulus space and individually diluted them to a point of about equal intensity. We then prepared various odorant mixtures, each containing various numbers of molecular components, and asked human participants to rate the perceptual similarity of such mixture pairs. We found that as we increased the number of nonoverlapping, equal-intensity components in odorant mixtures, the mixtures became more similar to each other, despite not having a single component in common. With similar to 30 components, most mixtures smelled alike. After participants were acquainted with a novel, arbitrarily named mixture of similar to 30 equal-intensity components, they later applied this name more readily to other novel mixtures of similar to 30 equal-intensity components spanning stimulus space, but not to mixtures containing fewer components or to mixtures that did not span stimulus space. We conclude that a common olfactory percept, "olfactory white," is associated with mixtures of similar to 30 or more equal-intensity components that span stimulus space, implying that olfactory representations are of features of molecules rather than of molecular identity.
-
(2012) PLoS One. 7, 9, 44272. Abstract
Whisking mediated touch is an active sense whereby whisker movements are modulated by sensory input and behavioral context. Here we studied the effects of touching an object on whisking in head-fixed rats. Simultaneous movements of whiskers C1, C2, and D1 were tracked bilaterally and their movements compared. During free-air whisking, whisker protractions were typically characterized by a single acceleration-deceleration event, whisking amplitude and velocity were correlated, and whisk duration correlated with neither amplitude nor velocity. Upon contact with an object, a second acceleration-deceleration event occurred in about 25% of whisk cycles, involving both contacting (C2) and non-contacting (C1, D1) whiskers ipsilateral to the object. In these cases, the rostral whisker (C2) remained in contact with the object throughout the double-peak phase, which effectively prolonged the duration of C2 contact. These "touch-induced pumps'' (TIPs) were detected, on average, 17.9 ms after contact. On a slower time scale, starting at the cycle following first touch, contralateral amplitude increased while ipsilateral amplitude decreased. Our results demonstrate that sensory-induced motor modulations occur at various timescales, and directly affect object palpation.
-
(2012) PLoS One. 7, 3, 33149. Abstract
The way information is represented by sequences of action potentials of spiking neurons is determined by the input each neuron receives, but also by its biophysics, and the specifics of the circuit in which it is embedded. Even the "code" of identified neurons can vary considerably from individual to individual. Here we compared the neural codes of the identified H1 neuron in the visual systems of two families of flies, blow flies and flesh flies, and explored the effect of the sensory environment that the flies were exposed to during development on the H1 code. We found that the two families differed considerably in the temporal structure of the code, its content and energetic efficiency, as well as the temporal delay of neural response. The differences in the environmental conditions during the flies' development had no significant effect. Our results may thus reflect an instance of a family-specific design of the neural code. They may also suggest that individual variability in information processing by this specific neuron, in terms of both form and content, is regulated genetically.
-
(2011) Nature Neuroscience. 14, 11, p. 1455-1461 Abstract
Organization of receptive surfaces reflects primary axes of perception. In vision, retinal coordinates reflect spatial coordinates. In audition, cochlear coordinates reflect tonal coordinates. However, the rules underlying the organization of the olfactory receptive surface are unknown. To test the hypothesis that organization of the olfactory epithelium reflects olfactory perception, we inserted an electrode into the human olfactory epithelium to directly measure odorant-induced evoked responses. We found that pairwise differences in odorant pleasantness predicted pairwise differences in response magnitude; that is, a location that responded maximally to a pleasant odorant was likely to respond strongly to other pleasant odorants, and a location that responded maximally to an unpleasant odorant was likely to respond strongly to other unpleasant odorants. Moreover, the extent of an individual's perceptual span predicted their span in evoked response. This suggests that, similarly to receptor surfaces for vision and audition, organization of the olfactory receptor surface reflects key axes of perception.
-
(2011) Journal of Neuroscience. 31, 44, p. 15732-15741 Abstract
The manner in which groups of neurons represent events in the external world is a central question in neuroscience. Estimation of the information encoded by small groups of neurons has shown that in many neural systems, cells carry mildly redundant information. These measures average over all the activity patterns of a neural population. Here, we analyze the population code of the salamander and guinea pig retinas by quantifying the information conveyed by specific multicell activity patterns. Synchronous spikes, even though they are relatively rare and highly informative, convey less information than the sum of either spike alone, making them redundant coding symbols. Instead, patterns of spiking in one cell and silence in others, which are relatively common and often overlooked as special coding symbols, were found to be mostly synergistic. Our results reflect that the mild average redundancy between ganglion cells that was previously reported is actually the result of redundant and synergistic multicell patterns, whose contributions partially cancel each other when taking the average over all patterns. We further show that similar coding properties emerge in a generic model of neural responses, suggesting that this form of combinatorial coding, in which specific compound patterns carry synergistic or redundant information, may exist in other neural circuits.
-
(2011) PLoS Computational Biology. 7, 9, 1002177. Abstract
Collective navigation and swarming have been studied in animal groups, such as fish schools, bird flocks, bacteria, and slime molds. Computer modeling has shown that collective behavior of simple agents can result from simple interactions between the agents, which include short range repulsion, intermediate range alignment, and long range attraction. Here we study collective navigation of bacteria-inspired smart agents in complex terrains, with adaptive interactions that depend on performance. More specifically, each agent adjusts its interactions with the other agents according to its local environment - by decreasing the peers' influence while navigating in a beneficial direction, and increasing it otherwise. We show that inclusion of such performance dependent adaptable interactions significantly improves the collective swarming performance, leading to highly efficient navigation, especially in complex terrains. Notably, to afford such adaptable interactions, each modeled agent requires only simple computational capabilities with short-term memory, which can easily be implemented in simple swarming robots.
-
(2011) Proceedings of the National Academy of Sciences of the United States of America. 108, 23, p. 9679-9684 Abstract
Information is carried in the brain by the joint activity patterns of large groups of neurons. Understanding the structure and function of population neural codes is challenging because of the exponential number of possible activity patterns and dependencies among neurons. We report here that for groups of similar to 100 retinal neurons responding to natural stimuli, pairwise-based models, which were highly accurate for small networks, are no longer sufficient. We show that because of the sparse nature of the neural code, the higher-order interactions can be easily learned using a novel model and that a very sparse low-order interaction network underlies the code of large populations of neurons. Additionally, we show that the interaction network is organized in a hierarchical and modular manner, which hints at scalability. Our results suggest that learnability may be a key feature of the neural code.
-
(2010) Proceedings of the National Academy of Sciences of the United States of America. 107, 32, p. 14419-14424 Abstract
In retina and in cortical slice the collective response of spiking neural populations is well described by "maximum-entropy" models in which only pairs of neurons interact. We asked, how should such interactions be organized to maximize the amount of information represented in population responses? To this end, we extended the linear-nonlinear-Poisson model of single neural response to include pairwise interactions, yielding a stimulus-dependent, pairwise maximum-entropy model. We found that as we varied the noise level in single neurons and the distribution of network inputs, the optimal pairwise interactions smoothly interpolated to achieve network functions that are usually regarded as discrete-stimulus decorrelation, error correction, and independent encoding. These functions reflected a trade-off between efficient consumption of finite neural bandwidth and the use of redundancy to mitigate noise. Spontaneous activity in the optimal network reflected stimulus-induced activity patterns, and single-neuron response variability overestimated network noise. Our analysis suggests that rather than having a single coding principle hardwired in their architecture, networks in the brain should adapt their function to changing noise and stimulus correlations.
-
(2010) Journal of Neuroscience. 30, 27, p. 9017-9026 Abstract
Odor identity is coded in spatiotemporal patterns of neural activity in the olfactory bulb. Here we asked whether meaningful olfactory information could also be read from the global olfactory neural population response. We applied standard statistical methods of dimensionality-reduction to neural activity from 12 previously published studies using seven different species. Four studies reported olfactory receptor activity, seven reported glomerulus activity, and one reported the activity of projection-neurons. We found two linear axes of neural population activity that accounted for more than half of the variance in neural response across species. The first axis was correlated with the total sum of odor-induced neural activity, and reflected the behavior of approach or withdrawal in animals, and odorant pleasantness in humans. The second and orthogonal axis reflected odorant toxicity across species. We conclude that in parallel with spatiotemporal pattern coding, the olfactory system can use simple global computations to read vital olfactory information from the neural population response.
-
(2009) arXiv. 0912.5409. Abstract
Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins. Here we use this equivalence to construct Ising models that describe the correlated spiking activity of populations of 40 neurons in the salamander retina responding to natural movies. We show that pairwise interactions between neurons account for observed higher-order correlations, and that for groups of 10 or more neurons pairwise interactions can no longer be regarded as small perturbations in an independent system. We then construct network ensembles that generalize the network instances observed in the experiment, and study their thermodynamic behavior and coding capacity. Based on this construction, we can also create synthetic networks of 120 neurons, and find that with increasing size the networks operate closer to a critical point and start exhibiting collective behaviors reminiscent of spin glasses. We examine closely two such behaviors that could be relevant for neural code: tuning of the network to the critical point to maximize the ability to encode diverse stimuli, and using the metastable states of the Ising Hamiltonian as neural code words.
-
Odorant Concentration Dependence in Electroolfactograms Recorded From the Human Olfactory Epithelium(2009) Journal of Neurophysiology. 102, 4, p. 2121-2130 Abstract
LElectroolfactograms (EOGs) are the summated generator potentials of olfactory receptor neurons measured directly from the olfactory epithelium. To validate the sensory origin of the human EOG, we set out to ask whether EOGs measured in humans were odorant concentration dependent. Each of 22 subjects (12 women, mean age = 23.3 yr) was tested with two odorants, either valeric acid and linalool (n = 12) or isovaleric acid and l-carvone (n = 10), each delivered at four concentrations diluted with warm (37°C) and humidified (80%) odorless air. In behavior, increased odorant concentration was associated with increased perceived intensity (all F > 5, all P < 0.001). In EOG, increased odorant concentration was associated with increased area under the EOG curve (all F > 8, all P < 0.001). These findings substantiate EOG as a tool for probing olfactory coding directly at the level of olfactory receptor neurons in humans.
-
(2009) Journal of Physics: International Workshop on Statistical-Mechanical Informatics. 197, 012020. Abstract
Most of our knowledge about how the brain encodes information conies from recordings of single neurons. However, computations in the brain are carried out by large groups of neurons. Modelling the joint activity of many interacting elements is computationally hard because of the large number of possible activity patterns and limited experimental data. Recently it was shown in several different neural systems that maximum entropy pairwise models, which rely only on tiring rates and pairwise correlations of neurons, are excellent models for the distribution of activity patterns of neural populations, and in particular, their responses to natural stimuli. Using simultaneous recordings of large groups of neurons in the vertebrate retina responding to naturalistic stimuli, we show here that the relevant statistics required for finding the pan-wise model can be accurately estimated within seconds. Furthermore, while higher order statistics may, in theory, improve model accuracy, they are, in practice, harmful for times of up to 20 minutes due to sampling noise. Finally, we demonstrate that trading accuracy for entropy may actually improve model performance when data is limited, and suggest an optimization method that automatically adjusts model constraints in order to achieve good performance.
-
(2007) Journal of Neurophysiology. 98, 3, p. 1380-1391 Abstract
The concerted action of saccades and fixational eye movements are crucial for seeing stationary objects in the visual world. We studied how these eye movements contribute to retinal coding of visual information using the archer fish as a model system. We quantified the animal's ability to distinguish among objects of different sizes and measured its eye movements. We recorded from populations of retinal ganglion cells with a multielectrode array, while presenting visual stimuli matched to the behavioral task. We found that the beginning of fixation, namely the time immediately after the saccade, provided the most visual information about object size, with fixational eye movements, which consist of tremor and drift in the archer fish, yielding only a minor contribution. A simple decoder that combined information from
-
(2006) arXiv. 0611072. Abstract
Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins. Here we use this equivalence to construct Ising models that describe the correlated spiking activity of populations of 40 neurons in the retina, and show that pairwise interactions account for observed higher-order correlations. By first finding a representative ensemble for observed networks we can create synthetic networks of 120 neurons, and find that with increasing size the networks operate closer to a critical point and start exhibiting collective behaviors reminiscent of spin glasses.
-
(2006) Nature. 440, 7087, p. 1007-1012 Abstract
Biological networks have so many possible states that exhaustive sampling is impossible. Successful analysis thus depends on simplifying hypotheses, but experiments on many systems hint that complicated, higher-order interactions among large groups of elements have an important role. Here we show, in the vertebrate retina, that weak correlations between pairs of neurons coexist with strongly collective behaviour in the responses of ten or more neurons. We find that this collective behaviour is described quantitatively by models that capture the observed pairwise correlations but assume no higher-order interactions. These maximum entropy models are equivalent to Ising models, and predict that larger networks are completely dominated by correlation effects. This suggests that the neural code has associative or error-correcting properties, and we provide preliminary evidence for such behaviour. As a first test for the generality of these ideas, we show that similar results are obtained from networks of cultured cortical neurons.
-
(2005) Neuron. 46, 3, p. 493-504 Abstract
We have explored the manner in which the population of retinal ganglion cells collectively represent the visual world. Ganglion cells in the salamander were recorded simultaneously with a multielectrode array during stimulation with both artificial and natural visual stimuli, and the mutual information that single cells and pairs of cells conveyed about the stimulus was estimated. We found significant redundancy between cells spaced as far as 500 mu m apart. When we used standard methods for defining functional types, only ON-type and OFF-type cells emerged as truly independent information channels. Although the average redundancy between nearby cell pairs was moderate, each ganglion cell shared information with many neighbors, so that visual information was represented 10-fold within the ganglion cell population. This high degree of retinal redundancy suggests that design principles beyond coding efficiency may be important at the population level.
-
(2003) Journal of Neuroscience. 23, 37, p. 11539-11553 Abstract
A key issue in understanding the neural code for an ensemble of neurons is the nature and strength of correlations between neurons and how these correlations are related to the stimulus. The issue is complicated by the fact that there is not a single notion of independence or lack of correlation. We distinguish three kinds: ( 1) activity independence; ( 2) conditional independence; and ( 3) information independence. Each notion is related to an information measure: the information between cells, the information between cells given the stimulus, and the synergy of cells about the stimulus, respectively. We show that these measures form an interrelated framework for evaluating contributions of signal and noise correlations to the joint information conveyed about the stimulus and that at least two of the three measures must be calculated to characterize a population code. This framework is compared with others recently proposed in the literature. In addition, we distinguish questions about how information is encoded by a population of neurons from how that information can be decoded. Although information theory is natural and powerful for questions of encoding, it is not sufficient for characterizing the process of decoding. Decoding fundamentally requires an error measure that quantifies the importance of the deviations of estimated stimuli from actual stimuli. Because there is no a priori choice of error measure, questions about decoding cannot be put on the same level of generality as for encoding.
-
(2003) Physical Review Letters. 91, 23, 238701. Abstract
Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible N-point correlation is measured by a decrease in entropy for the joint distribution of N variables relative to the maximum entropy allowed by all the observed N-1 variable distributions. We calculate the "connected information" terms for several examples and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source.
-
(2003) Advances in Neural Information Processing Systems 15. Obermayer K., Becker S. & Thrun S.(eds.). p. 213-220 Abstract
A population of neurons typically exhibits a broad diversity of responsesto sensory inputs. The intuitive notion of functional classification is thatcells can be clustered so that most of the diversity is captured by the identity of the clusters rather than by individuals within clusters. We showhow this intuition can be made precise using information theory, without any need to introduce a metric on the space of stimuli or responses.Applied to the retinal ganglion cells of the salamander, this approach recovers classical results, but also provides clear evidence for subclassesbeyond those identified previously. Further, we find that each of the ganglion cells is functionally unique, and that even within the same subclassonly a few spikes are needed to reliably distinguish between cells.
-
A new nonparametric pairwise clustering algorithm based on iterative estimation of distance profiles(2002) Machine Learning. 47, 1, p. 35-61 Abstract
We present a novel pairwise clustering method. Given a proximity matrix of pairwise relations (i.e. pairwise similarity or dissimilarity estimates) between data points, our algorithm extracts the two most prominent clusters in the data set. The algorithm, which is completely nonparametric, iteratively employs a two-step transformation on the proximity matrix. The first step of the transformation represents each point by its relation to all other data points, and the second step re-estimates the pairwise distances using a statistically motivated proximity measure on these representations. Using this transformation, the algorithm iteratively partitions the data points, until it finally converges to two clusters. Although the algorithm is simple and intuitive, it generates a complex dynamics of the proximity matrices. Based on this bipartition procedure we devise a hierarchical clustering algorithm, which employs the basic bipartition algorithm in a straightforward divisive manner. The hierarchical clustering algorithm copes with the model validation problem using a general cross-validation approach, which may be combined with various hierarchical clustering methods.We further present an experimental study of this algorithm. We examine some of the algorithm's properties and performance on some synthetic and 'standard' data sets. The experiments demonstrate the robustness of the algorithm and indicate that it generates a good clustering partition even when the data is noisy or corrupted.
-
(2001) Advances in Neural Information Processing Systems 13. Tresp, Dietterich TG. & Leen TK.(eds.). p. 159-165 Abstract
The problem of neural coding is to understand how sequences of action potentials (spikes) are related to sensory stimuli, motor outputs, or (ultimately) thoughts and intentions. One clear question is whether the same coding rules are used by different neurons, or by corresponding neurons in different individuals. We present a quantitative formulation of this problem using ideas from information theory, and apply this approach to the analysis of experiments in the fly visual system. We find significant individual differences in the structure of the code, particularly in the way that temporal patterns of spikes are used to convey information beyond that available from variations in spike rate. On the other hand, all the flies in our ensemble exhibit a high coding efficiency, so that every spike carries the same amount of information in all the individuals. Thus the neural code has a quantifiable mixture of individuality and universality.
-
(2000) Advances in Neural Information Processing Systems 12. Muller KR., Solla SA. & Leen TK.(eds.). Vol. 12. p. 178-184 Abstract
The reliability and accuracy of spike trains have been shown to depend on the nature of the stimulus that the neuron encodes. Adding ion channel stochasticity to neuronal models results in a macroscopic behavior that replicates the input-dependent reliability and precision of real neurons. We calculate the amount of information that an ion channel based stochastic Hodgkin-Huxley (HH) neuron model can encode about a wide set of stimuli. We show that both the information rate and the information per spike of the stochastic model are similar to the values reported experimentally Moreover, the amount of information that the neuron encodes is correlated with the amplitude of fluctuations in the input, and less so with the average firing rate of the neuron. We also show that for the HH ion channel density, the information capacity is robust to changes in the density of ion channels in the membrane, whereas changing the ratio between the Na+ and K+ ion channels has a considerable effect an the information that the neuron can encode. Finally, we suggest that neurons may maximize their information capacity by appropriately balancing the density of the different ion channels that underlie neuronal excitability.
-
(1999) Journal of Physiology Paris. 93, 4, p. 263-270 Abstract
Derailed models of single neurons are typically focused on the dendritic tree and ignore the axonal tree, assuming that the axon is a simple transmission line. In the last 40 years, however, several theoretical and experimental studies have suggested that axone could implement information processing tasks by exploiting: 1) the time delay in action potential (AP) propagation along the axon; 2) the differential filtering of APs into the axonal subtrees; and 3) their activity-dependent excitability. Models for axonal trees have attempted to examine the feasibility of these ideas. However, because the physiological and anatomical data on axons are seriously limited, realistic models of axone have not been developed. The present paper summarizes the main insights that were gained from simplified models of axons; it also highlights the stochastic nature of axone, a topic that was largely neglected in classical models of axons. The advance of new experimental techniques makes it now possible to pay a very close experimental visit to axons. Theoretical tools and fast computers enable to go beyond the simplified models and to construct realistic models of axons. When tightly linked, experiments and theory will help to unravel how axons share the information processing tasks that single neurons implement.
-
(1998) Neural Computation. 10, 7, p. 1679-1703 Abstract
The firing reliability and precision of an isopotential membrane patch consisting of a realistically large number of ion channels is investigated using a stochastic Hodgkin-Huxley (HH) model. In sharp contrast to the deterministic HH model, the biophysically inspired stochastic model reproduces qualitatively the different reliability and precision characteristics of spike firing in response to DC and fluctuating current input in neocortical neurons, as reported by Mainen & Sejnowski (1995). For DC inputs, spike timing is highly unreliable; the reliability and precision are significantly increased for fluctuating current input. This behavior is critically determined by the relatively small number of excitable channels that are opened near threshold for spike firing rather than by the total number of channels that exist in the membrane patch. Channel fluctuations, together with the inherent bistability in the HH equations, give rise to three additional experimentally observed phenomena: subthreshold oscillations in the membrane voltage for DC input, "spontaneous" spikes for subthreshold inputs, and "missing" spikes for suprathreshold inputs. We suggest that the noise inherent in the operation of ion channels enables neurons to act as "smart" encoders. Slowly varying, uncorrelated inputs are coded with low reliability and accuracy and, hence, the information about such inputs is encoded almost exclusively by the spike rate. On the other hand, correlated presynaptic activity produces sharp fluctuations in the input to the postsynaptic cell, which are then encoded with high reliability and accuracy. In this case, information about the input exists in the exact timing of the spikes. We conclude that channel stochasticity should be considered in realistic models of neurons.
-
(1997) Ophthalmology. 104, 4, p. 639-642 Abstract
Purpose: The early detection of visual threatening retinal thickness alterations is important for the purpose of offering affected patients proven treatment when indicated. Currently, the clinical methods that are available for obtaining an impression of retinal thickness are subjective and require experience and expertise. The authors present measurements of foveal thickness in healthy individuals obtained by a newly developed instrument that enables a noninvasive, noncontact, safe, and fast measurement of the retinal thickness anywhere in the posterior pole.Methods: A prototype of the retinal thickness analyzer (RTA) that operates on the principle of laser-slit biomicroscopy was used, Retinal thickness was measured in healthy emmetropic volunteers.Results: Fifty eyes were measured. The average thickness at the foveola was 178 +/- 44 mu m (+/- standard deviation; range, 100-322 mu m).Conclusion: The retinal thickness measured by the RTA in healthy subjects correspond well to previous published data (in vivo and histologic) on retinal thickness, This instrument may prove valuable in detecting retinal thickness alteration in macular diseases.