Proceedings Article10.1145/1273496.1273579
Dimensionality reduction and generalization
Sofia Mosci,Lorenzo Rosasco,Alessandro Verri +2 more
- 20 Jun 2007
- pp 657-664
30
TL;DR: This paper shows that performing KPCA and then ordinary least squares on the projected data is equivalent to spectral cut-off regularization, the regularization parameter being exactly the number of principal components to keep and proposes a parameter choice procedure allowing to prove consistency of the algorithm.
read more
Abstract: In this paper we investigate the regularization property of Kernel Principal Component Analysis (KPCA), by studying its application as a preprocessing step to supervised learning problems. We show that performing KPCA and then ordinary least squares on the projected data, a procedure known as kernel principal component regression (KPCR), is equivalent to spectral cut-off regularization, the regularization parameter being exactly the number of principal components to keep. Using probabilistic estimates for integral operators we can prove error estimates for KPCR and propose a parameter choice procedure allowing to prove consistency of the algorithm.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Nonlinear Dimensionality Reduction with Local Spline Embedding
TL;DR: An optimization framework based on reconstruction error analysis, which can yield a global optimum for nonlinear dimensionality reduction (NLDR), is developed and extended to embed out of samples via spline interpolation.
118
Embedding New Data Points for Manifold Learning via Coordinate Propagation
Shiming Xiang,Feiping Nie,Yangqiu Song,Changshui Zhang,Chunxia Zhang +4 more
- 01 Jan 2007
TL;DR: In this article, Tangent space projection and smooth splines are used to yield an initial coordinate for each new data point, according to their local geometrical relations, and an iterative algorithm for coordinate propagation is developed.
46
Embedding new data points for manifold learning via coordinate propagation
TL;DR: This work first formulate this task as a quadratic programming, and then develops an iterative algorithm for coordinate propagation, to propagate the known coordinates to each of the new data points.
44
•Journal Article
A Cure for Variance Inflation in High Dimensional Kernel Principal Component Analysis
TL;DR: It is shown that variance inflation is also present in kernel principal component analysis (kPCA) and a non-parametric renormalization scheme is provided which can quite efficiently restore generalizability in kPCA.
Linear Dimensionality Reduction for Margin-Based Classification: High-Dimensional Data and Sensor Networks
Kush R. Varshney,Alan S. Willsky +1 more
TL;DR: This work poses a joint optimization problem for linear dimensionality reduction and margin-based classification, and develops a coordinate descent algorithm on the Stiefel manifold for its solution, which enables it to extend for sensor networks with a message-passing approach requiring little communication.
32
References
•Book
The Elements of Statistical Learning
Trevor Hastie,Robert Tibshirani,Jerome H. Friedman +2 more
- 01 Jan 2001
29.4K
The Elements of Statistical Learning
TL;DR: Chapter 11 includes more case studies in other areas, ranging from manufacturing to marketing research, and a detailed comparison with other diagnostic tools, such as logistic regression and tree-based methods.
15.5K
Advances in kernel methods: support vector learning
Bernhard Schölkopf,Christopher John Burges,Alexander J. Smola +2 more
- 08 Feb 1999
TL;DR: Support vector machines for dynamic reconstruction of a chaotic system, Klaus-Robert Muller et al pairwise classification and support vector machines, Ulrich Kressel.
7.3K
Theory of Reproducing Kernels.
TL;DR: In this paper, a short historical introduction is given to indicate the different manners in which these kernels have been used by various investigators and discuss the more important trends of the application of these kernels without attempting, however, a complete bibliography of the subject matter.
•Book
Regularization of Inverse Problems
Heinz W. Engl,Martin Hanke,Andreas Neubauer +2 more
- 31 Jul 1996
TL;DR: Inverse problems have been studied in this article, where Tikhonov regularization of nonlinear problems has been applied to weighted polynomial minimization problems, and the Conjugate Gradient Method has been used for numerical realization.
5.7K