Journal Article10.1007/S10107-006-0706-8
Cubic regularization of Newton method and its global performance
Yurii Nesterov,Boris T. Polyak +1 more
1.2K
TL;DR: This paper provides theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem and proves general local convergence results for this scheme.
read more
Abstract: In this paper, we provide theoretical analysis for a cubic regularization of Newton method as applied to unconstrained minimization problem. For this scheme, we prove general local convergence results. However, the main contribution of the paper is related to global worst-case complexity bounds for different problem classes including some nonconvex cases. It is shown that the search direction can be computed by standard linear algebra technique.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
Stochastic Recursive Gradient Algorithm for Nonconvex Optimization
TL;DR: This paper studies and analyzes the mini-batch version of StochAstic Recursive grAdient algoritHm (SARAH), a method employing the stochastic recursive gradient, for solving empirical loss minimization for the case of nonconvex losses and provides a sublinear convergence rate and a linear convergence rate for gradient dominated functions.
Stochastic Recursive Gradient Algorithm for Nonconvex Optimization
Lam M. Nguyen,Jie Liu,Katya Scheinberg,Martin Takáč +3 more
TL;DR: This paper analyzes the mini-batch SARAH algorithm for nonconvex optimization, providing sublinear and linear convergence rates for general and gradient-dominated functions, respectively, outperforming other stochastic gradient algorithms for nonconvex losses.
Second-order optimization with lazy Hessians
TL;DR: The authors proposed to reuse a previously seen Hessian for several iterations while computing new gradients at each step of the method, which significantly reduces the overall arithmetical complexity of second-order optimization schemes.
Inexact Relative Smoothness and Strong Convexity for Optimization and Variational Inequalities by Inexact Model
Fedor Stonyakin,Alexander Tyurin,Alexander Gasnikov,Pavel Dvurechensky,Artem Agafonov,Darina Dvinskikh,Mohammad Alkousa,Dmitry Pasechnyuk,Sergei Artamonov,Victorya Piskunova +9 more
TL;DR: This paper proposes a general algorithmic framework for first-order optimization methods, including minimization, saddle-point problems, and variational inequalities, using inexact models to obtain known and new methods with optimal complexity.
Sample Complexity of Policy Gradient Finding Second-Order Stationary Points
Long Yang,Qian Zheng,Gang Pan +2 more
TL;DR: This paper investigates the sample complexity of policy gradient methods in reinforcement learning, showing that convergence to a second-order stationary point (SOSP) guarantees a local maximum, and achieving an (ε, √εχ)-SOSP in O(ε−9/2) samples with high probability.
References
A method for the solution of certain non – linear problems in least squares
TL;DR: In this article, the problem of least square problems with non-linear normal equations is solved by an extension of the standard method which insures improvement of the initial solution, which can also be considered an extension to Newton's method.
•Book
Practical Methods of Optimization
Roger Fletcher
- 01 Jan 2009
TL;DR: The aim of this book is to provide a Discussion of Constrained Optimization and its Applications to Linear Programming and Other Optimization Problems.
9.3K
•Book
Numerical methods for unconstrained optimization and nonlinear equations
John E. Dennis,Robert B. Schnabel +1 more
- 01 Mar 1983
TL;DR: Newton's Method for Nonlinear Equations and Unconstrained Minimization and methods for solving nonlinear least-squares problems with Special Structure.
8.2K