Classification using sparse representations: a biologically plausible approach
TL;DR: This work demonstrates that classification using sparse representations can be performed in a neurally plausible manner, and hence, that this mechanism of classification might be exploited by the brain.
read more
Abstract: Representing signals as linear combinations of basis vectors sparsely selected from an overcomplete dictionary has proven to be advantageous for many applications in pattern recognition, machine learning, signal processing, and computer vision. While this approach was originally inspired by insights into cortical information processing, biologically plausible approaches have been limited to exploring the functionality of early sensory processing in the brain, while more practical applications have employed non-biologically plausible sparse coding algorithms. Here, a biologically plausible algorithm is proposed that can be applied to practical problems. This algorithm is evaluated using standard benchmark tasks in the domain of pattern classification, and its performance is compared to a wide range of alternative algorithms that are widely used in signal and image processing. The results show that for the classification tasks performed here, the proposed method is competitive with the best of the alternative algorithms that have been evaluated. This demonstrates that classification using sparse representations can be performed in a neurally plausible manner, and hence, that this mechanism of classification might be exploited by the brain.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Figures
Figure 3: The total classification error over all three datasets for PC/BC-DIM and the 11 other algorithms under test. For each bar, the first two shaded segments correspond to the classification error for the USPS dataset, the second two segments show the the classification error for the ISOLET dataset, and the last two segments show the error for the MNIST dataset. In each of these pairs of shaded segments, the classification error on the test set is shown first, and the error on the training set is shown second. Note, however, that for many algorithms the classification error on the training set is zero so the second shaded segment in each pair is not visible. The first column shows the classification error when classification was performed by finding a sparse representation for the entire dictionary and then determining which class-specific sub-set of coefficients produced the lowest reconstruction error: similar to the method used in Wright et al. (2009b). These are the same values as shown in the first column of Fig. 2. The second column shows the classification error when classification was performed by finding a sparse representation for the entire dictionary and then determining which class-specific sub-set of coefficients had the highest sum. The third column shows the classification error when classification was performed by finding sparse representations separately for class-specific sub-dictionaries and determining which sub-dictionary produced the lowest reconstruction error: similar to the method used in Sprechmann and Sapiro (2010). The fourth column shows classification error calculated as for the third column, except in this case the algorithm under test had been used to learn the class-specific sub-dictionaries using the ILS-DLA dictionary learning method (Engan et al., 2007). 
Figure 2: Comparison of the proposed, biologically-plausible, sparse coding algorithm (PC/BC-DIM) with 11 alternative algorithms using the USPS (top-row), ISOLET (middle row), and MNIST (bottom-row) datasets. The first column shows the percentage of signals misclassified by each algorithm: the dark part of each bar shows the classification error on the test set, the light part of each bar shows the classification error on the training set. For most algorithms there is no light part of the bar as there was no classification error on the training set. The second column shows the normalised squared error between the signal and the reconstruction of the signal generated by the sparse representation. This value has been averaged across all exemplars in the test set. The third column shows the sparsity of the representation, calculated using Hoyer’s measure, averaged across all exemplars in the test set. The fourth column shows the total time taken by each algorithm to process all the exemplars in the test set. The execution time data is only provided for the USPS dataset. 
Figure 5: Scatter plots showing the relationship between classification error and (a) reconstruction error, and (b) sparsity. Each point shows results from a different sparse-coding algorithm when applied to classifying the test data from each dataset: USPS (squares), ISOLET (circles), and MNIST (crosses). The same results are also shown in Fig. 2. 
Table 1: The set of algorithms for finding sparse representations that are tested in this article. 
Figure 4: The total classification error over all three datasets for k-nearest neighbours classifiers, with varying values of k. The format of this figure is identical to, and described in the caption of, Fig. 3. 
Figure 1: The PC/BC-DIM neural network architecture. Rectangles represent populations of neurons, with y labelling the population of prediction neurons and e labelling the population of error-detecting neurons. Open arrows signify excitatory connections, filled arrows indicate inhibitory connections, crossed connections signify a many-to-many connectivity pattern between the neurons in two populations, and parallel connections indicate a one-to-one mapping of connections between the neurons in two populations. The feedback (or generative) weights, labelled V, correspond to a dictionary of basis vectors, and the activity of the prediction neurons, y, correspond to the coefficients of the sparse representation. The feedforward (or discriminative) weights, labelled W, are equal to a rescaled transpose of the feedback weights. Hence, if a pair of neurons in the e and y populations are connected by strong (or weak) feedforward weights, then the reciprocal feedback connection is also strong (or weak).
Citations
Predictive coding as a model of cognition
TL;DR: This article shows that predictive coding can simulate phenomena such as categorisation, the influence of abstract knowledge on perception, recall and reasoning about conceptual knowledge, context-dependent behavioural control, and naive physics.
Learning Receptive Fields and Quality Lookups for Blind Quality Assessment of Stereoscopic Images
TL;DR: A blind quality assessment for stereoscopic images is proposed by learning the characteristics of receptive fields (RFs) from perspective of dictionary learning, and constructing quality lookups to replace human opinion scores without performance loss.
46
A common network architecture efficiently implements a variety of sparsity-based inference problems
TL;DR: A wide variety of sparsity-based probabilistic inference problems proposed in the signal processing and statistics literatures can be implemented exactly in the common network architecture known as the locally competitive algorithm (LCA).
A structure optimization framework for feed-forward neural networks using sparse representation
TL;DR: A sparse-representation based framework, termed SRS, is introduced to generate a small-sized network structure without compromising the network performance and Experimental results indicate that the SRS framework performs favourably compared to alternative structure optimization algorithms.
20
A neural implementation of the Hough transform and the advantages of explaining away
TL;DR: A new method is proposed for implementing the voting process in the Hough transform using a competitive neural network algorithm to perform a form of probabilistic inference known as "explaining away", which results in a sparse accumulator array in which the parameter values of image features can be more accurately identified.
16
References
Robust Face Recognition via Sparse Representation
TL;DR: This work considers the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise, and proposes a general classification algorithm for (image-based) object recognition based on a sparse representation computed by C1-minimization.
$rm K$ -SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
TL;DR: A novel algorithm for adapting dictionaries in order to achieve sparse signal representations, the K-SVD algorithm, an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data.
10K
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
TL;DR: It is shown that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex.
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
Michael Elad,Michal Aharon +1 more
TL;DR: This work addresses the image denoising problem, where zero-mean white and homogeneous Gaussian additive noise is to be removed from a given image, and uses the K-SVD algorithm to obtain a dictionary that describes the image content effectively.
6.2K
Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.
Rajesh P. N. Rao,Dana H. Ballard +1 more
TL;DR: Results suggest that rather than being exclusively feedforward phenomena, nonclassical surround effects in the visual cortex may also result from cortico-cortical feedback as a consequence of the visual system using an efficient hierarchical strategy for encoding natural images.
5.4K