A Line Search Based Proximal Stochastic Gradient Algorithm with Dynamical Variance Reduction

TL;DR: In this article , the numerical solution of strictly convex unconstrained optimization problems by linesearch Newton-CG methods is studied and the expected iteration complexity bounds are derived for finite-sum minimization.

TL;DR: This paper introduces a variable metric proximal stochastic gradient method for supervised classification problems, incorporating automatic sample size selection and non-monotone line search, and provides convergence results for convex and non-convex objectives, outperforming state-of-the-art methods in numerical experiments.

TL;DR: In this paper , a more correct argument is employed to obtain the inequality (A11) from (A10), provided that a stronger hypothesis on the sequence $$\{\varepsilon _k\}$$ is included.

TL;DR: It is shown that various inverse problems in signal recovery can be formulated as the generic problem of minimizing the sum of two convex functions with certain regularity properties, which makes it possible to derive existence, uniqueness, characterization, and stability results in a unified and standardized fashion for a large class of apparently disparate problems.

TL;DR: The basic properties of proximity operators which are relevant to signal processing and optimization methods based on these operators are reviewed and proximal splitting methods are shown to capture and extend several well-known algorithms in a unifying framework.

TL;DR: This work was supported by the Agence Nationale de la Recherche under grants ANR-08-BLAN-0294-02 and ANR09-EMER-004-03.

TL;DR: An Introduction to Optimization, Second Edition helps students build a solid working knowledge of the field, including unconstrained optimization, linear programming, and constrained optimization.