Proceedings Article10.1145/1553374.1553417
Gradient descent with sparsification: an iterative algorithm for sparse recovery with restricted isometry property
Rahul Garg,Rohit Khandekar +1 more
- 14 Jun 2009
- pp 337-344
TL;DR: The Matlab implementation of GraDeS (Gradient Descent with Sparsification) outperforms previously proposed algorithms like Subspace Pursuit, StOMP, OMP, and Lasso by an order of magnitude and uncovered cases where L1-regularized regression (Lasso) fails but <b>GraDeS</b> finds the correct solution.
read more
Abstract: We present an algorithm for finding an s-sparse vector x that minimizes the square-error ∥y -- Φx∥2 where Φ satisfies the restricted isometry property (RIP), with isometric constant δ2s 1 and Hs sets all but s largest magnitude coordinates to zero. GraDeS converges to the correct solution in constant number of iterations. The condition δ2s
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Computational Methods for Sparse Solution of Linear Inverse Problems
Joel A. Tropp,Stephen J. Wright +1 more
- 29 Apr 2010
TL;DR: This paper surveys the major practical algorithms for sparse approximation with specific attention to computational issues, to the circumstances in which individual methods tend to perform well, and to the theoretical guarantees available.
Computational Methods for Sparse Solution of Linear Inverse Problems In many engineering areas, such as signal processing, practical results can be obtained by identifying approaches that yield the greatest quality improvement, or by selecting the most suitable computation methods.
Joel A. Tropp,Stephen J. Wright +1 more
- 01 Jan 2010
TL;DR: In this paper, a survey of the major practical algorithms for sparse approximation is presented, focusing on computational issues, circumstances in which individual methods tend to perform well, and theoretical guarantees available.
954
•Proceedings Article
TernGrad: ternary gradients to reduce communication in distributed deep learning
Wei Wen,Cong Xu,Feng Yan,Chunpeng Wu,Yandan Wang,Yi Chen,Hai Li +6 more
- 04 Dec 2017
TL;DR: This work mathematically proves the convergence of TernGrad under the assumption of a bound on gradients, and proposes layer-wise ternarizing and gradient clipping to improve its convergence.
Hard Thresholding Pursuit: An Algorithm for Compressive Sensing
TL;DR: A new iterative algorithm to find sparse solutions of underdetermined linear systems is introduced and it is shown that, under a certain condition on the restricted isometry constant of the matrix of the linear system, the Hard Thresholding Pursuit algorithm indeed finds all $s$-sparse solutions.
•Posted Content
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
TL;DR: TernGrad as discussed by the authors uses ternary gradients to accelerate distributed deep learning in data parallelism, which can reduce the communication cost of synchronizing gradients and parameters by ternarizing and gradient clipping.
630
References
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
TL;DR: In this paper, the authors considered the model problem of reconstructing an object from incomplete frequency samples and showed that with probability at least 1-O(N/sup -M/), f can be reconstructed exactly as the solution to the lscr/sub 1/ minimization problem.
Atomic Decomposition by Basis Pursuit
TL;DR: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
11.3K
An Introduction To Compressive Sampling
TL;DR: The theory of compressive sampling, also known as compressed sensing or CS, is surveyed, a novel sensing/sampling paradigm that goes against the common wisdom in data acquisition.
11.2K
Matching pursuits with time-frequency dictionaries
Stéphane Mallat,Zhifeng Zhang +1 more
TL;DR: The authors introduce an algorithm, called matching pursuit, that decomposes any signal into a linear expansion of waveforms that are selected from a redundant dictionary of functions, chosen in order to best match the signal structures.
10.2K
Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit
Joel A. Tropp,Anna C. Gilbert +1 more
TL;DR: It is demonstrated theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal.