Sharp Time–Data Tradeoffs for Linear Inverse Problems
TL;DR: In this paper, the authors characterized sharp time-data tradeoffs for optimization problems used for solving linear inverse problems and showed that a linear convergence rate is attainable even though the least square objective is not strongly convex in these settings.
read more
Abstract: In this paper, we characterize sharp time–data tradeoffs for optimization problems used for solving linear inverse problems. We focus on the minimization of a least-squares objective subject to a constraint defined as the sub-level set of a penalty function. We present a unified convergence analysis of the gradient projection algorithm applied to such problems. We sharply characterize the convergence rate associated with a wide variety of random measurement ensembles in terms of the number of measurements and structural complexity of the signal with respect to the chosen penalty function. The results apply to both convex and nonconvex constraints, demonstrating that a linear convergence rate is attainable even though the least squares objective is not strongly convex in these settings. When specialized to Gaussian measurements our results show that such linear convergence occurs when the number of measurements is merely four times the minimal number required to recover the desired signal at all (also known as the phase transition). We also achieve a slower but geometric rate of convergence precisely above the phase transition point. Extensive numerical results suggest that the derived rates exactly match the empirical performance.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview
Yuejie Chi,Yue Lu,Yuxin Chen +2 more
TL;DR: This tutorial-style overview highlights the important role of statistical models in enabling efficient nonconvex optimization with performance guarantees and reviews two contrasting approaches: two-stage algorithms, which consist of a tailored initialization step followed by successive refinement; and global landscape analysis and initialization-free algorithms.
542
Non-convex Optimization for Machine Learning
Prateek Jain,Purushottam Kar +1 more
TL;DR: Non-convex optimization as discussed by the authors is a generalization of the convex optimization problem, and it has been widely used in machine learning applications, such as deep learning and reinforcement learning.
516
Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation
Suhas Sreehari,Singanallur Venkatakrishnan,Brendt Wohlberg,Gregery T. Buzzard,Lawrence F. Drummy,Jeff Simmons,Charles A. Bouman +6 more
TL;DR: This paper presents an algorithm for electron tomographic reconstruction and sparse image interpolation that exploits the nonlocal redundancy in images, and demonstrates that the algorithm produces higher quality reconstructions on both simulated and real electron microscope data, along with improved convergence properties compared to other methods.
455
•Posted Content
Low-rank Solutions of Linear Matrix Equations via Procrustes Flow
TL;DR: Procrustes Flow as mentioned in this paper uses a thresholding scheme followed by gradient descent on a non-convex objective to recover a low-rank matrix from linear measurements and shows that as long as the measurements obey a standard restricted isometry property, their algorithm converges to the unknown matrix at a geometric rate.
276
State evolution for approximate message passing with non-separable functions
TL;DR: In this paper, the authors generalized state evolution to Lipschitz continuous non-separable nonlinearities, for Gaussian matrices, and introduced a modified algorithm called LAMP for Long AMP.
186
References
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
TL;DR: In this paper, the authors considered the model problem of reconstructing an object from incomplete frequency samples and showed that with probability at least 1-O(N/sup -M/), f can be reconstructed exactly as the solution to the lscr/sub 1/ minimization problem.
A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
Amir Beck,Marc Teboulle +1 more
TL;DR: A new fast iterative shrinkage-thresholding algorithm (FISTA) which preserves the computational simplicity of ISTA but with a global rate of convergence which is proven to be significantly better, both theoretically and practically.
14.3K
Atomic Decomposition by Basis Pursuit
TL;DR: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions.
11.3K
Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit
Joel A. Tropp,Anna C. Gilbert +1 more
TL;DR: It is demonstrated theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal.
Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering
TL;DR: An algorithm based on an enhanced sparse representation in transform domain based on a specially developed collaborative Wiener filtering achieves state-of-the-art denoising performance in terms of both peak signal-to-noise ratio and subjective visual quality.
9.6K