Open AccessProceedings Article
Gradient-based kernel method for feature extraction and variable selection
Kenji Fukumizu,Chenlei Leng +1 more
- 03 Dec 2012
- Vol. 25, pp 2114-2122
TL;DR: A novel kernel approach to dimension reduction for supervised learning: feature extraction and variable selection and in combination of a sparse penalty, the method is extended to variable selection, following the approach by Chen et al.
read more
Abstract: We propose a novel kernel approach to dimension reduction for supervised learning: feature extraction and variable selection; the former constructs a small number of features from predictors, and the latter finds a subset of predictors. First, a method of linear feature extraction is proposed using the gradient of regression function, based on the recent development of the kernel method. In comparison with other existing methods, the proposed one has wide applicability without strong assumptions on the regressor or type of variables, and uses computationally simple eigendecomposition, thus applicable to large data sets. Second, in combination of a sparse penalty, the method is extended to variable selection, following the approach by Chen et al. [2]. Experimental results show that the proposed methods successfully find effective features and variables without parametric models.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
Structure Adaptative Approach for Dimension Reduction
TL;DR: In this paper, the authors proposed a method of effective dimension reduction for a multi-index model which is based on iterative improvement of the family of average derivative estimates, and showed that in the case when the effective dimension m of the index space does not exceed 3, this space can be estimated with the rate n − 1/2 under mild assumptions on the model.
132
An EnKF-based scheme to optimize hyper-parameters and features for SVM classifier
TL;DR: A novel scheme to optimize hyper-parameters and features by using the Ensemble Kalman Filter (EnKF), which is an iterative optimization algorithm used for high-dimensional nonlinear systems, and proposes ensemble evolution to converge to the global optimum.
23
•Proceedings Article
Kernel feature selection via conditional covariance minimization
Jianbo Chen,Mitchell Stern,Martin J. Wainwright,Michael I. Jordan +3 more
- 04 Dec 2017
TL;DR: A method for feature selection that employs kernel-based measures of independence to find a subset of covariates that is maximally predictive of the response, and compares favorably with other state-of-the-art algorithms on a variety of synthetic and real data sets.
•Journal Article
Characteristic kernels and infinitely divisible distributions
Yu Nishiyama,Kenji Fukumizu +1 more
TL;DR: In this paper, the authors connect shift-invariant characteristic kernels to infinitely divisible distributions on road and show that any kernel given by a bounded, continuous, and symmetric probability density function (pdf) of an infinitely-divisible distribution on road is characteristic.
Kernel machine learning methods to handle missing responses with complex predictors. Application in modelling five-year glucose changes using distributional representations
TL;DR: In this paper , a new set of kernel methods were proposed to handle missing data in the response variables, which were applied to predict long-term changes in glycated haemoglobin (A1c), the primary biomarker used to diagnose and monitor the progression of diabetes mellitus, making emphasis on exploring the predictive potential of continuous glucose monitoring (CGM).
10
References
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Jianqing Fan,Runze Li +1 more
TL;DR: In this article, penalized likelihood approaches are proposed to handle variable selection problems, and it is shown that the newly proposed estimators perform as well as the oracle procedure in variable selection; namely, they work as well if the correct submodel were known.
Model selection and estimation in regression with grouped variables
Ming Yuan,Yi Lin +1 more
TL;DR: In this paper, instead of selecting factors by stepwise backward elimination, the authors focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection.
The adaptive lasso and its oracle properties
TL;DR: A new version of the lasso is proposed, called the adaptive lasso, where adaptive weights are used for penalizing different coefficients in the ℓ1 penalty, and the nonnegative garotte is shown to be consistent for variable selection.