EM algorithms without missing data.
TL;DR: A theoretical perspective clarifies the operation of the EM algorithm and suggests novel generalizations that lead to highly stable algorithms with well-understood local and global convergence properties in medical statistics.
read more
Abstract: Most problems in computational statistics involve optimization of an objective function such as a loglikelihood, a sum of squares, or a log posterior function. The EM algorithm is one of the most effective algorithms for maximization because it iteratively transfers maximization from a complex function to a simple, surrogate function. This theoretical perspective clarifies the operation of the EM algorithm and suggests novel generalizations. Besides simplifying maximization, optimization transfer usually leads to highly stable algorithms with well-understood local and global convergence properties. Although convergence can be excruciatingly slow, various devices exist for accelerating it. Beginning with the EM algorithm, we review in this paper several optimization transfer algorithms of substantial utility in medical statistics.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
A Tutorial on MM Algorithms
David R. Hunter,Kenneth Lange +1 more
TL;DR: The principle behind MM algorithms is explained, some methods for constructing them are suggested, and some of their attractive features are discussed.
Majorization-Minimization Algorithms in Signal Processing, Communications, and Machine Learning
TL;DR: An overview of the majorization-minimization (MM) algorithmic framework, which can provide guidance in deriving problem-driven algorithms with low computational cost and is elaborated by a wide range of applications in signal processing, communications, and machine learning.
1.7K
Optimization Transfer Using Surrogate Objective Functions
TL;DR: Because optimization transfer algorithms often exhibit the slow convergence of EM algorithms, two methods of accelerating optimization transfer are discussed and evaluated in the context of specific problems.
950
Quantile Regression via an MM Algorithm
David R. Hunter,Kenneth Lange +1 more
TL;DR: In this paper, an iterative algorithm for finding sample quantiles without sorting is presented, and a generalization of the algorithm to nonlinear quantile regression is explored, termed an MM, or majorize-minimize, algorithm.
Mixtures of conditional mean- and covariance-structure models
TL;DR: In this article, the authors consider mixtures of multivariate normals where the expected value for each component depends on possibly nonnormal regressor variables and the expected values and covariance matrices of the mixture components are parameterized using conditional mean-and covariance-structures.
206
References
Generalized Linear Models
John A. Nelder,R. W. M. Wedderburn +1 more
- 01 May 1972
TL;DR: In this paper, the authors used iterative weighted linear regression to obtain maximum likelihood estimates of the parameters with observations distributed according to some exponential family and systematic effects that can be made linear by a suitable transformation.
9.7K
•Book
Numerical methods for unconstrained optimization and nonlinear equations
John E. Dennis,Robert B. Schnabel +1 more
- 01 Mar 1983
TL;DR: Newton's Method for Nonlinear Equations and Unconstrained Minimization and methods for solving nonlinear least-squares problems with Special Structure.
8.2K
•Book
Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Classics in Applied Mathematics, 16)
John E. Dennis,Robert B. Schnabel +1 more
- 01 Feb 1996
TL;DR: In this paper, Schnabel proposed a modular system of algorithms for unconstrained minimization and nonlinear equations, based on Newton's method for solving one equation in one unknown convergence of sequences of real numbers.
6.8K
•Book
The EM algorithm and extensions
Geoffrey J. McLachlan,Thriyambakam Krishnan +1 more
- 15 Nov 1996
TL;DR: The EM Algorithm and Extensions describes the formulation of the EM algorithm, details its methodology, discusses its implementation, and illustrates applications in many statistical contexts, opening the door to the tremendous potential of this remarkably versatile statistical tool.
Related Papers (5)
David R. Hunter,Kenneth Lange +1 more
Geoffrey J. McLachlan,Thriyambakam Krishnan +1 more
- 15 Nov 1996