Open AccessProceedings Article
Sparse probabilistic projections
Cédric Archambeau,Francis Bach +1 more
- 08 Dec 2008
- Vol. 21, pp 73-80
TL;DR: This work presents a generative model for performing sparse probabilistic projections, which includes sparse principal component analysis and sparse canonical correlation analysis as special cases, and derives a variational Expectation-Maximisation algorithm for the estimation of the hyperparameters.
read more
Abstract: We present a generative model for performing sparse probabilistic projections, which includes sparse principal component analysis and sparse canonical correlation analysis as special cases. Sparsity is enforced by means of automatic relevance determination or by imposing appropriate prior distributions, such as generalised hyperbolic distributions. We derive a variational Expectation-Maximisation algorithm for the estimation of the hyperparameters and show that our novel probabilistic approach compares favourably to existing techniques. We illustrate how the proposed method can be applied in the context of cryptoanalysis as a preprocessing tool for the construction of template attacks.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
A Survey on Canonical Correlation Analysis
TL;DR: This survey targets to provide a well-organized overview for CCA and its extensions from the perspective of both model formation and model optimization, and provides several promising future research directions that can improve the current state of the art.
157
•Proceedings Article
Factorized Orthogonal Latent Spaces
Mathieu Salzmann,Carl Henrik Ek,Raquel Urtasun,Trevor Darrell +3 more
- 31 Mar 2010
TL;DR: This paper proposes a robust approach to factorizing the latent space into shared and private spaces by introducing orthogonality constraints, which penalize redundant latent representations.
142
Group Factor Analysis
TL;DR: Group Factor Analysis (GFA) as mentioned in this paper is an extension of canonical correlation analysis to more than two sets, in a way that is more flexible than previous extensions, and it is formulated as a variational inference of a latent variable model with structural sparsity.
Multivariate multi-way analysis of multi-source data
TL;DR: The applicability area of multivariate, multi-way ANOVA-type methods to multi-source cases is extended by introducing a novel Bayesian model capable of finding covariate-related dependencies between the sources and to source-specific ones.
Nonparametric bayesian sparse factor models with application to gene expression modelling
TL;DR: In this paper, a nonparametric Bayesian extension of Factor Analysis (FA) is proposed where observed data Y is modeled as a linear superposition, G, of a potentially infinite number of hidden factors, X. The Indian Buffet Process (IBP) is used as a prior on G to incorporate sparsity and to allow the number of latent features to be inferred.
References
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Jianqing Fan,Runze Li +1 more
TL;DR: In this article, penalized likelihood approaches are proposed to handle variable selection problems, and it is shown that the newly proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well if the correct submodel were known.
Probabilistic Principal Component Analysis
TL;DR: In this paper, the principal axes of a set of observed data vectors may be determined through maximum-likelihood estimation of parameters in a latent variable model closely related to factor analysis.
Sparse Principal Component Analysis
TL;DR: This work introduces a new method called sparse principal component analysis (SPCA) using the lasso (elastic net) to produce modified principal components with sparse loadings and shows that PCA can be formulated as a regression-type optimization problem.
A view of the EM algorithm that justifies incremental, sparse, and other variants
Radford M. Neal,Geoffrey E. Hinton +1 more
- 26 Mar 1998
TL;DR: In this paper, an incremental variant of the EM algorithm is proposed, in which the distribution for only one of the unobserved variables is recalculated in each E step, which is shown empirically to give faster convergence in a mixture estimation problem.
2.6K