Newton's Method

TL;DR: Two main approaches to Newton's method for unconstrained minimization are presented: the line search approach and the trust region approach and it is shown how quasi-Newton methods can be derived quite naturally from Newton's methods.

Abstract: Newton's method plays a central role in the development of numerical techniques for optimization In fact, most of the current practical methods for optimization can be viewed as variations on Newton's method It is therefore important to understand Newton's method as an algorithm in its own right and as a key introduction to the most recent ideas in this area One of the aims of this expository paper is to present and analyze two main approaches to Newton's method for unconstrained minimization: the line search approach and the trust region approach The other aim is to present some of the recent developments in the optimization field which are related to Newton's method In particular, we explore several variations on Newton's method which are appropriate for large scale problems, and we also show how quasi-Newton methods can be derived quite naturally from Newton's method