Open AccessProceedings Article
A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
Sahand Negahban,Bin Yu,Martin J. Wainwright,Pradeep Ravikumar +3 more
- 07 Dec 2009
- Vol. 22, pp 1348-1356
TL;DR: A unified framework for establishing consistency and convergence rates for regularized M-estimators under high-dimensional scaling is provided and one main theorem is state and shown how it can be used to re-derive several existing results, and also to obtain several new results.
read more
Abstract: High-dimensional statistical inference deals with models in which the the number of parameters p is comparable to or larger than the sample size n. Since it is usually impossible to obtain consistent procedures unless p/n → 0, a line of recent work has studied models with various types of structure (e.g., sparse vectors; block-structured matrices; low-rank matrices; Markov assumptions). In such settings, a general approach to estimation is to solve a regularized convex program (known as a regularized M-estimator) which combines a loss function (measuring how well the model fits the data) with some regularization function that encourages the assumed structure. The goal of this paper is to provide a unified framework for establishing consistency and convergence rates for such regularized M-estimators under high-dimensional scaling. We state one main theorem and show how it can be used to re-derive several existing results, and also to obtain several new results on consistency and convergence rates. Our analysis also identifies two key properties of loss and regularization functions, referred to as restricted strong convexity and decomposability, that ensure the corresponding regularized M-estimators have fast convergence rates.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
Robust Confidence Intervals in High-Dimensional Left-Censored Regression
Jelena Bradic,Jiaqi Guo +1 more
TL;DR: In this paper, the authors developed smoothed estimating equations that augment the de-biasing method, such that the resulting estimator is adaptive to censoring and is more robust to the misspecification of the error distribution.
1
•Posted Content
A proximal dual semismooth Newton method for computing zero-norm penalized QR estimator
TL;DR: Numerical comparisons show that MSCRA\_PPA has comparable estimation performance with the latter two methods and requires only half (respectively, one-third) of the time required by MSC RA\_ADMM and MSCra\_IPM.
1
Factorisable Sparse Tail Event Curves with Expectiles
TL;DR: Oberwolfach as mentioned in this paper reported new developments in functional and highly multivariate statistical methodologies. But their focus was not on statistical methods, but on the application of functional models.
1
•Posted Content
Sparse recovery via nonconvex regularized $M$-estimators over $\ell_q$-balls
TL;DR: In this paper, the recovery properties of non-convex regularized $M$-estimators, under the assumption that the true parameter is of soft sparsity, were analyzed.
1
•Posted Content
Sparse estimation via 𝓁 q optimization method in high-dimensional linear regression.
TL;DR: A general $q-restricted eigenvalue condition (REC) is introduced and its sufficient conditions are provided in terms of several widely-used regularity conditions such as sparse eigen value condition, restricted isometry property, and mutual incoherence property to exhibit the stable recovery property of the optimization methods.
References
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Atomic Decomposition by Basis Pursuit
TL;DR: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
11.3K
Model selection and estimation in regression with grouped variables
Ming Yuan,Yi Lin +1 more
TL;DR: In this paper, instead of selecting factors by stepwise backward elimination, the authors focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection.
Decoding by linear programming
Emmanuel J. Candès,Terence Tao +1 more
TL;DR: F can be recovered exactly by solving a simple convex optimization problem (which one can recast as a linear program) and numerical experiments suggest that this recovery procedure works unreasonably well; f is recovered exactly even in situations where a significant fraction of the output is corrupted.
Exact Matrix Completion via Convex Optimization
TL;DR: It is proved that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries, and that objects other than signals and images can be perfectly reconstructed from very limited information.