Open AccessProceedings Article
Fast Iterative Kernel PCA
Nicol N. Schraudolph,Simon Günter,S. V. N. Vishwanathan +2 more
- 04 Dec 2006
- Vol. 19, pp 1225-1232
TL;DR: Two methods to improve convergence of the Kernel Hebbian Algorithm for iterative kernel PCA by incorporating the reciprocal of the current estimated eigenvalues as a gain vector and derive and apply Stochastic Meta-Descent (SMD) to KHA/et.
read more
Abstract: We introduce two methods to improve convergence of the Kernel Hebbian Algorithm (KHA) for iterative kernel PCA. KHA has a scalar gain parameter which is either held constant or decreased as 1/t, leading to slow convergence. Our KHA/et algorithm accelerates KHA by incorporating the reciprocal of the current estimated eigenvalues as a gain vector. We then derive and apply Stochastic Meta-Descent (SMD) to KHA/et; this further speeds convergence by performing gain adaptation in RKHS. Experimental results for kernel PCA and spectral clustering of USPS digits as well as motion capture and image de-noising problems confirm that our methods converge substantially faster than conventional KHA.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Fast Iterative Kernel Principal Component Analysis
TL;DR: Gain adaptation methods that improve convergence of the kernel Hebbian algorithm for iterative kernel PCA by incorporating the reciprocal of the current estimated eigenvalues as part of a gain vector in reproducing kernel Hilbert space to further speed up convergence.
Online Kernel Principal Component Analysis: A Reduced-Order Model
TL;DR: This paper examines a kernel-based version of Oja's rule, initially put forward to extract a linear principal axe, and derives a recursive algorithm to discover the first principal axis, and extend it to multiple axes, to provide an online scheme for kernel-PCA.
Optimal Wind Clustering Methodology for Adequacy Evaluation in System Generation Studies Using Nonsequential Monte Carlo Simulation
TL;DR: In this paper, several clustering algorithms are investigated in order to group together wind parks with close statistical behavior and to integrate them in a realistic way into a nonsequential Monte Carlo adequacy evaluation process.
53
Machine learning algorithms, applications, and practices in data science
Kalidas Yeturu
- 01 Jan 2020
TL;DR: Data science is about end to end development of a smart solution that involves creation of pipelines for activities for data generation, business decision making and solution maintenance with humans in loop.
48
•Proceedings Article
Scale up nonlinear component analysis with doubly stochastic gradients
Bo Xie,Yingyu Liang,Le Song +2 more
- 07 Dec 2015
TL;DR: In this article, the authors proposed a simple, computationally efficient, and memory friendly algorithm based on the "doubly stochastic gradients" to scale up a range of kernel nonlinear component analysis, such as kernel PCA, CCA and SVD.
References
Backpropagation applied to handwritten zip code recognition
Yann LeCun,Bernhard E. Boser,John S. Denker,D. Henderson,Richard Howard,W. Hubbard,Lawrence D. Jackel +6 more
TL;DR: This paper demonstrates how constraints from the task domain can be integrated into a backpropagation network through the architecture of the network, successfully applied to the recognition of handwritten zip code digits provided by the U.S. Postal Service.
12.5K
A Stochastic Approximation Method
Herbert Robbins,Sutton Monro +1 more
TL;DR: In this article, a method for making successive experiments at levels x1, x2, ··· in such a way that xn will tend to θ in probability is presented.
•Proceedings Article
On Spectral Clustering: Analysis and an algorithm
Andrew Y. Ng,Michael I. Jordan,Yair Weiss +2 more
- 03 Jan 2001
TL;DR: A simple spectral clustering algorithm that can be implemented using a few lines of Matlab is presented, and tools from matrix perturbation theory are used to analyze the algorithm, and give conditions under which it can be expected to do well.
Nonlinear component analysis as a kernel eigenvalue problem
TL;DR: A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.
Optimal Unsupervised Learning in a Single-Layer Linear Feedforward Neural Network
TL;DR: An optimality principle is proposed which is based upon preserving maximal information in the output units and an algorithm for unsupervised learning based upon a Hebbian learning rule, which achieves the desired optimality is presented.
1.6K