Abstract: Non-negative matrix factorization (NMF) is a very efficient parameter-free method for decomposing multivariate data into strictly positive activations and basis vectors. However, the method is not suited for overcomplete representations, where usually sparse coding paradigms apply. We show how to merge the concepts of non-negative factorization with sparsity conditions. The result is a multiplicative algorithm that is comparable in efficiency to standard NMF, but that can be used to gain sensible solutions in the overcomplete cases. This is of interest e.g. for the case of learning and modeling of arrays of receptive fields arranged in a visual processing map, where an overcomplete representation is unavoidable.

Abstract: The human nervous system processes a continuous stream of multi-modal input from a rapidly changing environment. A key challenge for neural modeling is to explain how the neural microcircuits (columns, minicolumns, etc.) in the cerebral cortex whose anatomical and physiological structure is quite similar in many brain areas and species achieve this enormous computational task. We propose a computational model that could explain the potentially universal computational capabilities and does not require a task-dependent construction of neural circuits. Instead it is based on principles of high dimensional dynamical systems in combination with statistical learning theory, and can be implemented on generic evolved or found recurrent circuitry. This new approach towards understanding neural computation on the micro-level also suggests new ways of modeling cognitive processing in larger neural systems. In particular it questions traditional ways of thinking about neural coding.

Abstract: The ability of rats to use their whiskers for fine tactile discrimination rivals that of humans using their fingertips. Rats perform discriminations rapidly and accurately while palpating the environment with their whiskers. This suggests that whisker deflections produce a robust and reliable neural code. Whisker primary afferents respond with highly reproducible temporal spike patterns to transient stimuli. Here we show that, with the use of a linear kernel, any of these reproducible response trains recorded from an individual neuron can reliably predict complex whisker deflections. These predictions are significantly improved by integrating responses from neurons with opposite angular preferences.

Abstract: This paper surveys recent findings in neuroscience regarding the behavioral relevancy of the precise timing with which real spiking neurons emit spikes. The literature suggests that in almost any system where the processing-speed of a neural (sub)-system is required to be high, the timing of single spikes can be very precise and reliable. Additionally, new, more refined methods are finding precisely timed spikes where previously none where found. This line of evidence thus provides additional motivation for researching the computational properties of networks of artificial spiking neurons that compute with more precisely timed spikes.

Abstract: We present a system for adaptive spectral basis decomposition that learns to identify independent spectral features given a sequence of short-term Fourier spectra When applied to recordings of polyphonic piano music, the individual notes are identified as salient features, and hence each short-term spectrum is decomposed into a sum of note spectra; the resulting encoding can be used as a basis for polyphonic transcription The system is based on a probabilistic model equivalent to a form of noisy independent component analysis (ICA) or sparse coding with non-negativity constraints We introduce a novel modification to this model that recognises that a short-term Fourier spectrum can be thought of as a noisy realisation of the power spectral density of an underlying Gaussian process, where the noise is essentially multiplicative and non-Gaussian Results are presented for an analysis of a live recording of polyphonic piano music

Abstract: Direction-selective neurons in the primary visual cortex (V1) and the extrastriate motion area MT/V5 constitute a critical channel that links early cortical mechanisms of spatiotemporal integration to downstream signals that underlie motion perception. We studied how temporal integration in direction-selective cells depends on speed, spatial frequency (SF), and contrast using randomly moving sinusoidal gratings and spike-triggered average (STA) analysis. The window of temporal integration revealed by the STAs varied substantially with stimulus parameters, extending farther back in time for slow motion, high SF, and low contrast. At low speeds and high SF, STA peaks were larger, indicating that a single spike often conveyed more information about the stimulus under conditions in which the mean firing rate was very low. The observed trends were similar in V1 and MT and offer a physiological correlate for a large body of psychophysical data on temporal integration. We applied the same visual stimuli to a model of motion detection based on oriented linear filters (a motion energy model) that incorporated an integrate-and-fire mechanism and found that it did not account for the neuronal data. Our results show that cortical motion processing in V1 and in MT is highly nonlinear and stimulus dependent. They cast considerable doubt on the ability of simple oriented filter models to account for the output of direction-selective neurons in a general manner. Finally, they suggest that spike rate tuning functions may miss important aspects of the neural coding of motion for stimulus conditions that evoke low firing rates.

Abstract: The present study explores the issue of cortical coding by spike count and timing using statistical and information theoretic methods. We have shown in previous studies that neurons in the auditory...

Abstract: A rate code assumes that a neuron's response is completely characterized by its time-varying mean firing rate. This assumption has successfully described neural responses in many systems. The noise in rate coding neurons can be quantified by the coherence function or the correlation coefficient between the neuron's deterministic time-varying mean rate and noise corrupted single spike trains. Because of the finite data size, the mean rate cannot be known exactly and must be approximated. We introduce novel unbiased estimators for the measures of coherence and correlation which are based on the extrapolation of the signal to noise ratio in the neural response to infinite data size. We then describe the application of these estimates to the validation of the class of stimulus-response models that assume that the mean firing rate captures all the information embedded in the neural response. We explain how these quantifiers can be used to separate response prediction errors that are due to inaccurate model assumptions from errors due to noise inherent in neuronal spike trains.

Abstract: We measured multi-channel MEG (MagnetoEncephaloGraphy) responses to a set of color stimuli (for twelve colors). Subjects are gazing at colored squares during the measurements. Each ECD (Estimated Current Dipole) in the brain, thus localized response to the corresponding color stimulus, is moving inside area V4α (BARTELS and ZEKI, 2000), showing a characteristic orbital path. In our measurements, absolute positional- errors are still large. However, the relative resolution is quite high, since the orbits of ECDs are localized in quite a small volume (1 (cm 3 )). This localized volume corresponds to V4α. ECDs in the volume are forming different orbits from each other. V4α conserves topological relation in the chromatic coordinate. Our results suggest that the neural coding of chromaticity topologically corresponds to well known geometrical coding of chromaticity in terms of a continual deformation between neighboring representations. Therefore, the observations could be interpreted as that the chromatic topology is represented in V4α and that the refined coding of each color is spatiotemporally represented within V4α. This fine spatiotemporal resolution in the results is obtained by our non-parametric gICA (geometric Independent Component Analysis) algorithm based on the isotropy condition of connections among adjacent tangential spaces in the neural activities.

Abstract: A rate code assumes that a neuron's response is completely characterized by its time-varying mean firing rate. This assumption has successfully described neural responses in many systems. The noise in rate coding neurons can be quantified by the coherence function or the correlation coefficient between the neuron's deterministic time-varying mean rate and noise corrupted single spike trains. Because of the finite data size, the mean rate cannot be known exactly and must be approximated. We introduce novel unbiased estimators for the measures of coherence and correlation which are based on the extrapolation of the signal to noise ratio in the neural response to infinite data size. We then describe the application of these estimates to the validation of the class of stimulus-response models that assume that the mean firing rate captures all the information embedded in the neural response. We explain how these quantifiers can be used to separate response prediction errors that are due to inaccurate model assumptions from errors due to noise inherent in neuronal spike trains.

Abstract: It is well known that some neurons tend to fire packets of action potentials followed by periods of quiescence (bursts) while others within the same stage of sensory processing fire in a tonic manner. However, the respective computational advantages of bursting and tonic neurons for encoding time varying signals largely remain a mystery. Weakly electric fish use cutaneous electroreceptors to convey information about sensory stimuli and it has been shown that some electroreceptors exhibit bursting dynamics while others do not. In this study, we compare the neural coding capabilities of tonically firing and bursting electroreceptor model neurons using information theoretic measures. We find that both bursting and tonically firing model neurons efficiently transmit information about the stimulus. However, the decoding mechanisms that must be used for each differ greatly: a non-linear decoder would be required to extract all the available information transmitted by the bursting model neuron whereas a linear one might suffice for the tonically firing model neuron. Further investigations using stimulus reconstruction techniques reveal that, unlike the tonically firing model neuron, the bursting model neuron does not encode the detailed time course of the stimulus. A novel measure of feature detection reveals that the bursting neuron signals certain stimulus features. Finally, we show that feature extraction and stimulus estimation are mutually exclusive computations occurring in bursting and tonically firing model neurons, respectively. Our results therefore suggest that stimulus estimation and feature extraction might be parallel computations in certain sensory systems rather than being sequential as has been previously proposed.

Abstract: We consider the dependence of information transfer by neurons on the Type I vs. Type II classification of their dynamics. Our computational study is based on Type I and II implementations of the Morris-Lecar model. It mainly concerns neurons, such as those in the auditory or electrosensory system, which encode band-limited amplitude modulations of a periodic carrier signal, and which fire at random cycles yet preferred phases of this carrier. We first show that the Morris-Lecar model with additive broadband noise ("synaptic noise") can exhibit such firing patterns with either Type I or II dynamics, with or without amplitude modulations of the carrier. We then compare the encoding of band-limited random amplitude modulations for both dynamical types. The comparison relies on a parameter calibration that closely matches firing rates for both models across a range of parameters. In the absence of synaptic noise, Type I performs slightly better than Type II, and its performance is optimal for perithreshold signals. However, Type II performs well over a slightly larger range of inputs, and this range lies mostly in the subthreshold region. Further, Type II performs marginally better than Type I when synaptic noise, which yields more realistic baseline firing patterns, is present in both models. These results are discussed in terms of the tuning and phase locking properties of the models with deterministic and stochastic inputs.

Abstract: The present essay concisely analyses the contemporary neurobiological debate concerning the hypothesis of the “temporal correlation” advanced to solve the perceptual problem of linking different features in a unitary object or visual scene. Although fascinating and grounded on simulations and brain models, in addition to important electrophysiological findings on the sensory systems, this hypothesis is regarded as not conclusive, and it still excites numerous critical observations from different approaches. Nevertheless, it has contributed to an innovative use of the idea of cortical oscillations, as regards its usual employment in reference to the electrical activity of the brain. It also opens a new perspective in the assumption of the temporal pattern to read the neural code.

Abstract: Sensory neurons in many brain areas spike with precise timing to stimuli with temporal structures, and encode temporally complex stimuli into spatiotemporal spikes. How the downstream neurons read out such neural code is an important unsolved problem. In this paper, we describe a decoding scheme using a spiking recurrent neural network. The network consists of excitatory neurons that form a synfire chain, and two globally inhibitory interneurons of different types that provide delayed feedforward and fast feedback inhibition, respectively. The network signals recognition of a specific spatiotemporal sequence when the last excitatory neuron down the synfire chain spikes, which happens if and only if that sequence was present in the input spike stream. The recognition scheme is invariant to variations in the intervals between input spikes within some range. The computation of the network can be mapped into that of a finite state machine. Our network provides a simple way to decode spatiotemporal spikes with diverse types of neurons.

Abstract: We create a framework based on Fisher information for determining the most effective population coding scheme for representing a continuous-valued stimulus attribute over its entire range. Using this scheme, we derive optimal single- and multi-neuron rate codes for homogeneous populations using several statistical models frequently used to describe neural data. We show that each neuron's discharge rate should increase quadratically with the stimulus and that statistically independent neural outputs provides optimal coding. Only cooperative populations can achieve this condition in an informationally effective way.

Abstract: Many time-series in engineering arise from a sparse mixture of individual components. Sparse coding can be used to decompose such signals into a set of functions. Most sparse coding algorithms divide the signal into blocks. The functions learned from these blocks are, however, not independent of the temporal alignment of the blocks. We present a fast algorithm for sparse coding that does not depend on the block location. To reduce the dimensionality of the problem, a subspace selection step is used during signal decomposition. Due to this reduction, an iterative reweighted least squares method can be used for the constrained optimisation. We demonstrate the algorithm's abilities by learning functions from a polyphonic piano recording. The found functions represent individual notes and a sparse signal decomposition leads to a transcription of the piano signal.

Abstract: Publisher Summary This chapter demonstrates the applicability of physiological interpretations of behavioral categories for a less complex nervous system The studies of compound stimulus processing and learning that have focused on olfactory learning in an invertebrate—the honeybee Apis mellifera is described Honeybees are a traditional model for studying learning and memory Olfactory learning in bees is well characterized and follows the Pavlovian conditioning scheme A series of conditioning experiments is described and conclusions derived on how bees treat olfactory compound stimuli are presented in the chapter A neurobiological perspective is presented in which the behavioral findings are correlated with findings on neural coding of olfactory compound stimuli in the olfactory neuropils of the bee brain The chapter addresses the question of whether the nervous system treats and learns a sensory compound stimulus as the simple sum of its components or as an entity different from them by using olfactory discrimination and learning in honeybees The findings using optophysiological recordings from the antennal lobe of the bee are reviewed The local inhibitory network of the antennal lobe leads to nonlinear summation and suppression effects such that the neural representation of a mixture stimulus includes the representations of the components but also odor-specific inhibitory phenomena, which may correspond to the unique cue

Abstract: Images can be coded accurately using a sparse set of vectors from an overcomplete dictionary, with potential applications in image compression and feature selection for pattern recognition. We discuss algorithms that perform sparse coding and make three contributions. First, we compare our overcomplete dictionary learning algorithm (FOCUSS-CNDL) with overcomplete independent component analysis (ICA). Second, noting that once a dictionary has been learned in a given domain the problem becomes one of choosing the vectors to form an accurate, sparse representation, we compare a recently developed algorithm (sparse Bayesian learning with adjustable variance Gaussians) to well known methods of subset selection: matching pursuit and FOCUSS. Third, noting that in some cases it may be necessary to find a non-negative sparse coding, we present a modified version of the FOCUSS algorithm that can find such non-negative codings

Abstract: Shaped by evolutionary processes, sensory systems often represent behaviorally relevant stimuli with higher fidelity than other stimuli. The stimulus dependence of neural reliability could therefore provide an important clue in a search for relevant sensory signals. We explore this relation and introduce a novel iterative algorithm that allows one to find stimuli that are reliably represented by the sensory system under study. To assess the quality of a neural representation, we use stimulus reconstruction methods. The algorithm starts with the presentation of an initial stimulus (e.g. white noise). The evoked spike train is recorded and used to reconstruct the stimulus online. Within a closed-loop setup, this reconstruction is then played back to the sensory system. Iterating this procedure, the newly generated stimuli can be better and better reconstructed. We demonstrate the feasibility of this method by applying it to auditory receptor neurons in locusts. Our data show that the optimal stimuli often exhibit pronounced sub-threshold periods that are interrupted by short, yet intense pulses. Similar results are obtained for simple model neurons and suggest that these stimuli are encoded with high reliability by a large class of neurons.

Abstract: The reproducibility of neural spike train responses to an identical stimulus across different presentations (trials) has been studied extensively. Reliability, the degree of reproducibility of spike trains, was found to depend in part on the amplitude and frequency content of the stimulus [J. Hunter and J. Milton, J. Neurophysiol. 90, 387 (2003)]. The responses across different trials can sometimes be interpreted as the response of an ensemble of similar neurons to a single stimulus presentation. How does the reliability of the activity of neural ensembles affect information transmission between different cortical areas? We studied a model neural system consisting of two ensembles of neurons with Hodgkin-Huxley-type channels. The first ensemble was driven by an injected sinusoidal current that oscillated in the gamma-frequency range (40 Hz) and its output spike trains in turn drove the second ensemble by fast excitatory synaptic potentials with short term depression. We determined the relationship between the reliability of the first ensemble and the response of the second ensemble. In our paradigm the neurons in the first ensemble were initially in a chaotic state with unreliable and imprecise spike trains. The neurons became entrained to the oscillation and responded reliably when the stimulus power was increased by less than 10%. The firing rate of the first ensemble increased by 30%, whereas that of the second ensemble could increase by an order of magnitude. We also determined the response of the second ensemble when its input spike trains, which had non-Poisson statistics, were replaced by an equivalent ensemble of Poisson spike trains. The resulting output spike trains were significantly different from the original response, as assessed by the metric introduced by Victor and Purpura [J. Neurophysiol. 76, 1310 (1996)]. These results are a proof of principle that weak temporal modulations in the power of gamma-frequency oscillations in a given cortical area can strongly affect firing rate responses downstream by way of reliability in spite of rather modest changes in firing rate in the originating area.

Abstract: Mutual Information (MI) is a long studied measure of coding efficiency, and many attempts to apply it to population coding have been made. However, this is a computationally intractable task, and most previous studies redefine the criterion in forms of approximations. Recently we described properties of a simple lower bound on MI [2]. Here we describe the bound optimization procedure for learning of population codes in a simple point neural model. We compare our approach with other techniques maximizing approximations of MI, focusing on a comparison with the Fisher Information criterion.