Open AccessProceedings Article
On autoencoder scoring
Hanna Kamyshanska,Roland Memisevic +1 more
- 16 Jun 2013
- pp 720-728
TL;DR: This paper shows how an autoencoder can assign meaningful scores to data independently of training procedure and without reference to any probabilistic model, by interpreting it as a dynamical system and how one can combine multiple, unnormalized scores into a generative classifier.
read more
Abstract: Autoencoders are popular feature learning models because they are conceptually simple, easy to train and allow for efficient inference and training. Recent work has shown how certain autoencoders can assign an unnormalized "score" to data which measures how well the autoencoder can represent the data. Scores are commonly computed by using training criteria that relate the autoencoder to a probabilistic model, such as the Restricted Boltzmann Machine. In this paper we show how an autoencoder can assign meaningful scores to data independently of training procedure and without reference to any probabilistic model, by interpreting it as a dynamical system. We discuss how, and under which conditions, running the dynamical system can be viewed as performing gradient descent in an energy function, which in turn allows us to derive a score via integration. We also show how one can combine multiple, unnormalized scores into a generative classifier.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
A Projection Pursuit Algorithm for Exploratory Data Analysis
Jerome H. Friedman,John W. Tukey +1 more
TL;DR: An algorithm for the analysis of multivariate data is presented and is discussed in terms of specific examples to find one-and two-dimensional linear projections of multivariable data that are relatively highly revealing.
A review of unsupervised feature learning and deep learning for time-series modeling ☆
TL;DR: This paper overviews the particular challenges present in time-series data and provides a review of the works that have either applied time- series data to unsupervised feature learning algorithms or alternatively have contributed to modifications of featurelearning algorithms to take into account the challenges present.
1.3K
A practical tutorial on autoencoders for nonlinear feature fusion: taxonomy, models, software and guidelines
David Charte,Francisco Charte,Salvador García,María José del Jesus,Francisco Herrera,Francisco Herrera +5 more
TL;DR: Autoencoders (AEs) as mentioned in this paper have emerged as an alternative to manifold learning for conducting nonlinear feature fusion, and they can be used to generate reduced feature sets through the fusion of the original ones.
290
•Posted Content
Dense Associative Memory for Pattern Recognition
Dmitry Krotov,John J. Hopfield +1 more
TL;DR: In this paper, the authors proposed a duality between associative memory and neural networks commonly used in deep learning, and demonstrated the utility of the dense memories for two test cases: logical gate XOR and the recognition of handwritten digits from the MNIST data set.
213
•Posted Content
Zero-bias autoencoders and the benefits of co-adapting features
TL;DR: This work shows that negative biases are a natural result of using a hidden layer whose responsibility is to both represent the input data and act as a selection mechanism that ensures sparsity of the representation and proposes a new activation function that decouples the two roles of the hidden layer.
83
References
ImageNet classification with deep convolutional neural networks
TL;DR: A large, deep convolutional neural network was trained to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes and employed a recently developed regularization method called "dropout" that proved to be very effective.
•Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton +2 more
- 03 Dec 2012
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
•Book
Elements of information theory
Thomas M. Cover,Joy A. Thomas +1 more
- 01 Jan 1991
TL;DR: The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment.
Extracting and composing robust features with denoising autoencoders
Pascal Vincent,Hugo Larochelle,Yoshua Bengio,Pierre-Antoine Manzagol +3 more
- 05 Jul 2008
TL;DR: This work introduces and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern.
Training products of experts by minimizing contrastive divergence
TL;DR: A product of experts (PoE) is an interesting candidate for a perceptual system in which rapid inference is vital and generation is unnecessary because it is hard even to approximate the derivatives of the renormalization term in the combination rule.