Open AccessProceedings Article
A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
Sahand Negahban,Bin Yu,Martin J. Wainwright,Pradeep Ravikumar +3 more
- 07 Dec 2009
- Vol. 22, pp 1348-1356
TL;DR: A unified framework for establishing consistency and convergence rates for regularized M-estimators under high-dimensional scaling is provided and one main theorem is state and shown how it can be used to re-derive several existing results, and also to obtain several new results.
read more
Abstract: High-dimensional statistical inference deals with models in which the the number of parameters p is comparable to or larger than the sample size n. Since it is usually impossible to obtain consistent procedures unless p/n → 0, a line of recent work has studied models with various types of structure (e.g., sparse vectors; block-structured matrices; low-rank matrices; Markov assumptions). In such settings, a general approach to estimation is to solve a regularized convex program (known as a regularized M-estimator) which combines a loss function (measuring how well the model fits the data) with some regularization function that encourages the assumed structure. The goal of this paper is to provide a unified framework for establishing consistency and convergence rates for such regularized M-estimators under high-dimensional scaling. We state one main theorem and show how it can be used to re-derive several existing results, and also to obtain several new results on consistency and convergence rates. Our analysis also identifies two key properties of loss and regularization functions, referred to as restricted strong convexity and decomposability, that ensure the corresponding regularized M-estimators have fast convergence rates.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Sharp Time–Data Tradeoffs for Linear Inverse Problems
TL;DR: In this paper, the authors characterized sharp time-data tradeoffs for optimization problems used for solving linear inverse problems and showed that a linear convergence rate is attainable even though the least square objective is not strongly convex in these settings.
An analysis of penalized interaction models
Junlong Zhao,Chenlei Leng +1 more
TL;DR: A unified analysis on the convergence rate for a class of estimators, when the design satisfies the restricted eigenvalue condition is presented and it is shown that with probability tending to one, the resulting estimates have a rate of convergence in the error.
•Posted Content
The Highest Dimensional Stochastic Blockmodel with a Regularized Estimator
Karl Rohe,Tai Qin,Haoyang Fan +2 more
TL;DR: This is the first paper to explicitly introduce and demonstrate the advantages of statistical regularization in a parametric form for network analysis in the high dimensional Stochastic Block model.
Community Detection from Low-Rank Excitations of a Graph Filter
Hoi-To Wai,Santiago Segarra,Asuman Ozdaglar,Anna Scaglione,Ali Jadbabaie +4 more
- 15 Apr 2018
TL;DR: This paper provides theoretical bounds on the error introduced by this sketching procedure based on spectral features of the graph filter involved, and proposes to detect the communities by applying spectral clustering on the low-rank output covariance matrix.
Model Selection with Low Complexity Priors
TL;DR: In this paper, a general framework to capture the low-dimensional structure of a linear manifold is proposed, which includes convex, non-negative, closed and finite-valued functions that will promote objects living on lowdimensional subspaces.
References
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
Atomic Decomposition by Basis Pursuit
TL;DR: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
11.3K
Model selection and estimation in regression with grouped variables
Ming Yuan,Yi Lin +1 more
TL;DR: In this paper, instead of selecting factors by stepwise backward elimination, the authors focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection.
Decoding by linear programming
Emmanuel J. Candès,Terence Tao +1 more
TL;DR: F can be recovered exactly by solving a simple convex optimization problem (which one can recast as a linear program) and numerical experiments suggest that this recovery procedure works unreasonably well; f is recovered exactly even in situations where a significant fraction of the output is corrupted.
Exact Matrix Completion via Convex Optimization
TL;DR: It is proved that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries, and that objects other than signals and images can be perfectly reconstructed from very limited information.