Open AccessProceedings Article
A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
Sahand Negahban,Bin Yu,Martin J. Wainwright,Pradeep Ravikumar +3 more
- 07 Dec 2009
- Vol. 22, pp 1348-1356
TL;DR: A unified framework for establishing consistency and convergence rates for regularized M-estimators under high-dimensional scaling is provided and one main theorem is state and shown how it can be used to re-derive several existing results, and also to obtain several new results.
read more
Abstract: High-dimensional statistical inference deals with models in which the the number of parameters p is comparable to or larger than the sample size n. Since it is usually impossible to obtain consistent procedures unless p/n → 0, a line of recent work has studied models with various types of structure (e.g., sparse vectors; block-structured matrices; low-rank matrices; Markov assumptions). In such settings, a general approach to estimation is to solve a regularized convex program (known as a regularized M-estimator) which combines a loss function (measuring how well the model fits the data) with some regularization function that encourages the assumed structure. The goal of this paper is to provide a unified framework for establishing consistency and convergence rates for such regularized M-estimators under high-dimensional scaling. We state one main theorem and show how it can be used to re-derive several existing results, and also to obtain several new results on consistency and convergence rates. Our analysis also identifies two key properties of loss and regularization functions, referred to as restricted strong convexity and decomposability, that ensure the corresponding regularized M-estimators have fast convergence rates.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
A constrained \(\ell \)1 minimization approach for estimating multiple sparse Gaussian or nonparanormal graphical models
TL;DR: A novel theoretical proof is provided showing that SIMULE achieves a consistent result at the rate of O(\log (Kp)/n_{tot})$$O(log(Kp/ntot), and SIMULE shows significant improvement over state-of-the-art multi-sGGM and single-UGM baselines.
L'intelligence économique et la gestion des connaissances au sein des organisations
Audrey Knauf,Stéphane Goria +1 more
- 04 Oct 2010
TL;DR: In this paper, the authors define a categorie specifique d'organisations which prend la forme de dispositifs territoriaux d'appui aux entreprises avec pour principal objectif : l'aide a la decision par, notamment, des services personnalises de fourniture d'informations elaborees.
•Posted Content
High-Dimensional Semiparametric Selection Models: Estimation Theory with an Application to the Retail Gasoline Market
TL;DR: In this paper, a multi-stage projection-based Lasso procedure for the semiparametric sample selection model in high-dimensional settings under a weak nonparametric restriction on the selection correction is proposed.
•Proceedings Article
Towards faster rates and oracle property for low-rank matrix estimation
Huan Gui,Jiawei Han,Quanquan Gu +2 more
- 19 Jun 2016
TL;DR: In this paper, a unified framework for low-rank matrix estimation with nonconvex penalty is presented, and a proximal gradient homotopy algorithm is developed to solve the proposed optimization problem.
•Posted Content
Minimax Estimation of Conditional Moment Models
TL;DR: We develop an approach for estimating models described via conditional moment restrictions, with a prototypical application being non-parametric instrumental variable regression.
References
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Atomic Decomposition by Basis Pursuit
TL;DR: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
11.3K
Model selection and estimation in regression with grouped variables
Ming Yuan,Yi Lin +1 more
TL;DR: In this paper, instead of selecting factors by stepwise backward elimination, the authors focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection.
Decoding by linear programming
Emmanuel J. Candès,Terence Tao +1 more
TL;DR: F can be recovered exactly by solving a simple convex optimization problem (which one can recast as a linear program) and numerical experiments suggest that this recovery procedure works unreasonably well; f is recovered exactly even in situations where a significant fraction of the output is corrupted.
Exact Matrix Completion via Convex Optimization
TL;DR: It is proved that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries, and that objects other than signals and images can be perfectly reconstructed from very limited information.