Open AccessProceedings Article
Variational Sparse Coding
Francesco Tonolini,Bjørn Sand Jensen,Roderick Murray-Smith +2 more
- 25 Jul 2019
- pp 690-700
TL;DR: A model based on variational auto-encoders in which interpretation is induced through latent space sparsity with a mixture of Spike and Slab distributions as prior is proposed and provides unique capabilities, such as recovering feature exploitation, synthesising samples that share attributes with a given input object and controlling both discrete and continuous features upon generation.
read more
Abstract: Unsupervised discovery of interpretable features and controllable generation with highdimensional data are currently major challenges in machine learning, with applications
in data visualisation, clustering and artificial
data synthesis. We propose a model based
on variational auto-encoders (VAEs) in which
interpretation is induced through latent space
sparsity with a mixture of Spike and Slab distributions as prior. We derive an evidence
lower bound for this model and propose a specific training method for recovering disentangled features as sparse elements in latent vectors. In our experiments, we demonstrate superior disentanglement performance to standard
VAE approaches when an estimate of the number of true sources of variation is not available
and objects display different combinations of
attributes. Furthermore, the new model provides unique capabilities, such as recovering
feature exploitation, synthesising samples that
share attributes with a given input object and
controlling both discrete and continuous features upon generation.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
Disentangling Disentanglement in Variational Autoencoders.
TL;DR: In this paper, a generalization of disentanglement in VAEs is proposed, which is based on decomposition of the latent representation, i.e., the latent encodings of the data having an appropriate level of overlap, represented through the prior.
83
Good practices for Bayesian Optimization of high dimensional structured spaces
Eero Siivola,Javier González,Andrei Paleyes,Aki Vehtari +3 more
- 01 Jun 2021
TL;DR: In this paper, the effect of different search space design choices for performing Bayesian optimization in high dimensional structured datasets is studied, in particular, the influence of the dimensionality of the latent space, the role of the acquisition function and evaluate new methods to automatically define the optimization bounds.
36
•Posted Content
End-to-end Sinkhorn Autoencoder with Noise Generator
TL;DR: This work proposes a novel end-to-end Sinkhorn Autoencoder with a noise generator for efficient data collection simulation that outperforms conventional approaches on a challenging dataset of simulation data from Zero Degree Calorimeters of ALICE experiment in LHC.
12
•Posted Content
Convergence Rates of Variational Inference in Sparse Deep Learning.
TL;DR: The authors showed that variational inference for sparse deep learning retains the same generalization properties than exact Bayesian inference, and highlighted the connection between estimation and approximation theories via the classical bias-variance trade-off.
8
End-to-End Sinkhorn Autoencoder With Noise Generator
Kamil Rafal Deja,Jan Dubinski,Piotr Nowak,Sandro Christian Wenzel,Przemsysaw Spurek,Tomasz Trzcinski +5 more
TL;DR: In this paper, an end-to-end Sinkhorn autoencoder with a noise generator is proposed for efficient data collection simulation, which can explicitly align distribution of encoded real data examples and generated noise.
References
•Proceedings Article
Auto-Encoding Variational Bayes
Diederik P. Kingma,Max Welling +1 more
- 01 Jan 2014
TL;DR: A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.
Representation Learning: A Review and New Perspectives
TL;DR: Recent work in the area of unsupervised feature learning and deep learning is reviewed, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks.
•Posted Content
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
TL;DR: Fashion-MNIST is intended to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms, as it shares the same image size, data format and the structure of training and testing splits.
9.4K
•Posted Content
Decision-Making with Auto-Encoding Variational Bayes
TL;DR: This work describes the error of importance sampling as a function of posterior variance and shows that proposal distributions learned with evidence upper bounds are better than the current state of the art.
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
TL;DR: It is shown that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex.