Journal Article10.1007/S10107-006-0706-8
Cubic regularization of Newton method and its global performance
Yurii Nesterov,Boris T. Polyak +1 more
1.2K
TL;DR: This paper provides theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem and proves general local convergence results for this scheme.
read more
Abstract: In this paper, we provide theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem. For this scheme, we prove general local convergence results. However, the main contribution of the paper is related to global worst-case complexity bounds for different problem classes including some nonconvex cases. It is shown that the search direction can be computed by standard linear algebra technique.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Linear Speedup in Saddle-Point Escape for Decentralized Non-Convex Optimization
Stefan Vlaski,Ali H. Sayed +1 more
- 04 May 2020
TL;DR: The results imply that a linear speedup can be expected in the pursuit of second-order stationary points, which exclude local maxima as well as strict saddle-points and correspond to local or even global minima in many important learning settings.
1
•Posted Content
Finding Second-Order Stationary Point for Nonconvex-Strongly-Concave Minimax Problem
Luo Luo,Cheng Chen +1 more
TL;DR: In this article, the authors consider the smooth minimax optimization problem without convex-concave assumption and propose a nonasymptotic convergence behavior of finding second-order stationary points.
1
Convergence Rates of Stochastic Zeroth-order Gradient Descent for \L ojasiewicz Functions
Tianyu Wang,Yasong Feng +1 more
- 31 Oct 2022
TL;DR: In this article , the convergence rate of stochastic Zeroth-order Gradient Descent (SZGD) algorithm for Lojasiewicz functions was shown to converge faster than any other algorithm.
1
Approximate Secular Equations for the Cubic Regularization Subproblem
Yihang Gao,Man-Chung Yue,Michael K. Ng +2 more
- 27 Sep 2022
TL;DR: A novel CRS solver based on an approximate secular equation, which requires only some of the Hessian eigenvalues and is therefore much more efficient, which makes it particularly suitable for high-dimensional applications of unconstrained non-convex optimization, such as low-rank recovery and deep learning.
1
Augmented Newton Method for Optimization: Global Linear Rate and Momentum Interpretation
TL;DR: In this article , two variants of the Newton method, namely the penalty Newton method and the augmented Lagrangian method, are proposed for solving unconstrained minimization problems. And they provide global convergence results for the proposed methods under mild assumptions that hold for a wide variety of problems.
1
References
A method for the solution of certain non – linear problems in least squares
TL;DR: In this article, the problem of least square problems with non-linear normal equations is solved by an extension of the standard method which insures improvement of the initial solution, which can also be considered an extension to Newton's method.
•Book
Practical Methods of Optimization
Roger Fletcher
- 01 Jan 2009
TL;DR: The aim of this book is to provide a Discussion of Constrained Optimization and its Applications to Linear Programming and Other Optimization Problems.
9.3K
•Book
Numerical methods for unconstrained optimization and nonlinear equations
John E. Dennis,Robert B. Schnabel +1 more
- 01 Mar 1983
TL;DR: Newton's Method for Nonlinear Equations and Unconstrained Minimization and methods for solving nonlinear least-squares problems with Special Structure.
8.2K