Journal Article10.1007/S10107-006-0706-8
Cubic regularization of Newton method and its global performance
Yurii Nesterov,Boris T. Polyak +1 more
1.2K
TL;DR: This paper provides theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem and proves general local convergence results for this scheme.
read more
Abstract: In this paper, we provide theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem. For this scheme, we prove general local convergence results. However, the main contribution of the paper is related to global worst-case complexity bounds for different problem classes including some nonconvex cases. It is shown that the search direction can be computed by standard linear algebra technique.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Gradient Descent with Random Initialization: Fast Global Convergence for Nonconvex Phase Retrieval
TL;DR: In this article, the authors investigated the efficiency of gradient descent designed for the nonconvex least squares problem and showed that under Gaussian designs, gradient descent yields an accurate solution in O(log n+log(1/ε) ) iterations given nearly minimal samples, thus achieving near optimal computational and sample complexities at once.
41
•Posted Content
Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty form
TL;DR: It is shown that all approximate local optima are global optima for the penalty formulation of appropriately rank-constrained SDPs as long as the number of constraints scales sub-quadratically with the desired rank of the optimal solution.
40
•Proceedings Article
Online Learning with Non-Convex Losses and Non-Stationary Regret.
Xiand Gao,Xiaobo Li,Shuzhong Zhang +2 more
- 01 Jan 2018
TL;DR: A sublinear regret bound is established for online learning with non-convex loss functions and non-stationary regret measure by establishing a cumulative regret bound of O( √ T + VTT ), where VT is the total temporal variations of the loss functions.
•Proceedings Article
Efficiently escaping saddle points on manifolds
Chris Criscitiello,Nicolas Boumal +1 more
- 25 Jul 2019
TL;DR: The key technical idea is to generalize perturbed Riemannian gradient descent with a distinction between two types of gradient steps: "steps on the manifold" and "perturbed steps in a tangent space of the manifold."
•Posted Content
Structured Quasi-Newton Methods for Optimization with Orthogonality Constraints
TL;DR: In this article, a structured quasi-Newton method was proposed to approximate the original objective function in the Euclidean space and preserve the orthogonality constraints without performing the so-called vector transports.
38
References
A method for the solution of certain non – linear problems in least squares
TL;DR: In this article, the problem of least square problems with non-linear normal equations is solved by an extension of the standard method which insures improvement of the initial solution, which can also be considered an extension to Newton's method.
•Book
Practical Methods of Optimization
Roger Fletcher
- 01 Jan 2009
TL;DR: The aim of this book is to provide a Discussion of Constrained Optimization and its Applications to Linear Programming and Other Optimization Problems.
9.3K
•Book
Numerical methods for unconstrained optimization and nonlinear equations
John E. Dennis,Robert B. Schnabel +1 more
- 01 Mar 1983
TL;DR: Newton's Method for Nonlinear Equations and Unconstrained Minimization and methods for solving nonlinear least-squares problems with Special Structure.
8.2K