What is the kth iterate of this classical algorithm?

If µk1/2 is the kth iterate of this classical algorithm, then the next iterate isµk+11/2 = ∑n i=1 w k i yi∑ni=1 w k i, (1)where wki = |yi − µk1/2| −1.

Why is the objective function L() difficult to minimize?

The objective function L(θ) in (9) is difficult to minimize because it can admit multiple minima and because the underlying function ρq(r) is nondifferentiable at r = 0.

What is the technical conditionlim for q?

To ensure that a regression quantile actually exists, the authors impose the technical conditionlim ‖θ‖→∞ n∑ i=1 fi(θ)2 = ∞. (9)For q = 1/2, quantile regression is least absolute deviation, or L1, regression.

What is the criterion used for the MM algorithm?

Because several of their examples involve an objective function which converges in value to zero, the authors use criterion (22) for their examples.

What is the argument used in the proof of Proposition 4?

This preserves continuity of the θk 7→ θk+1 map, and it follows by the same argument used in the proof of Proposition 4 that all limit points of the modified algorithm are stationary points of L (θ).

What is the q quantile of an integrable random variable?

The q quantile of an integrable random variable Y minimizes the function E [ρq(Y − µ)], whereρq(r) = |r| [ q1{r≥0} + (1− q)1{r<0} ] = qr − r1{r<0}. (7)Although long known, this fact has “languished in the status of curiosum—appearing for example as an exercise in Ferguson (1967, p. 51),” as Koenker and Bassett (1978) put it.

Open AccessJournal Article10.1080/10618600.2000.10474866

Quantile Regression via an MM Algorithm

Q: What are the contributions mentioned in the paper "Quantile regression via an mm algorithm" ?

The current paper first presents an iterative algorithm for finding sample quantiles without sorting and then explores a generalization of the algorithm to nonlinear quantile regression. Their quantile regression algorithm is termed an MM, or Majorize-Minimize, algorithm because it entails majorizing the objective function by a quadratic function followed by minimizing that quadratic. The algorithm is conceptually simple and easy to code, and their numerical tests suggest that it is computationally competitive with a recent interior point algorithm for most problems.

Q: What is the best-known example of an MM algorithm?

The best-known example of an MM algorithm is the EM algorithm for maximum likelihood estimation in the presence of missingdata (Dempster et al., 1977).

Q: What are the main strengths of the MM algorithm presented in this paper?

The major strengths of the MM algorithm presented in this paper are its conceptual simplicity, ease of implementation, and numerical stability, qualities it shares with most other MM algorithms.

Q: What is the approach to the minimization problem?

Their approach to this minimization problem is first to construct a function that approximates L(θ) very closely and then to use an MM algorithm to minimize the approximating function.

David R. Hunter, +1 more

- 01 Mar 2000

- Journal of Computational and Graphical S...

- Vol. 9, Iss: 1, pp 60-77

363

TL;DR: In this paper, an iterative algorithm for finding sample quantiles without sorting is presented, and a generalization of the algorithm to nonlinear quantile regression is explored, termed an MM, or majorize-minimize, algorithm.

Abstract: Quantile regression is an increasingly popular method for estimating the quantiles of a distribution conditional on the values of covariates. Regression quantiles are robust against the influence of outliers and, taken several at a time, they give a more complete picture of the conditional distribution than a single estimate of the center. This article first presents an iterative algorithm for finding sample quantiles without sorting and then explores a generalization of the algorithm to nonlinear quantile regression. Our quantile regression algorithm is termed an MM, or majorize—minimize, algorithm because it entails majorizing the objective function by a quadratic function followed by minimizing that quadratic. The algorithm is conceptually simple and easy to code, and our numerical tests suggest that it is computationally competitive with a recent interior point algorithm for most problems.

AI Agents for this Paper

Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps

Most frequently asked questions

1. What are the contributions mentioned in the paper "Quantile regression via an mm algorithm" ?

The current paper first presents an iterative algorithm for finding sample quantiles without sorting and then explores a generalization of the algorithm to nonlinear quantile regression.. Their quantile regression algorithm is termed an MM, or Majorize-Minimize, algorithm because it entails majorizing the objective function by a quadratic function followed by minimizing that quadratic.. The algorithm is conceptually simple and easy to code, and their numerical tests suggest that it is computationally competitive with a recent interior point algorithm for most problems.

2. What is the best-known example of an MM algorithm?

The best-known example of an MM algorithm is the EM algorithm for maximum likelihood estimation in the presence of missingdata (Dempster et al., 1977).

3. What are the main strengths of the MM algorithm presented in this paper?

The major strengths of the MM algorithm presented in this paper are its conceptual simplicity, ease of implementation, and numerical stability, qualities it shares with most other MM algorithms.

4. What is the approach to the minimization problem?

Their approach to this minimization problem is first to construct a function that approximates L(θ) very closely and then to use an MM algorithm to minimize the approximating function.

Figure 1: The lower, piecewise linear function is the objective function for the 1/4 quantile of the points {1, 3, 4, 8, 10}. The upper curve is the majorizing surrogate function Q(µ | µk) corresponding to µk = 6.

Table 1: Thousands of FLOPs required until convergence and final objective function values L̂ for test problems of Koenker and Park (1996). In each cell, the MM results are on the first line and the interior point results are on the second line.

Figure 2: Continuing the example depicted in Figure 1, L(µ) is a piecewise linear objective function (a), L (µ) is the approximant (b) to the objective function, and Q (µ | µk) is the surrogate (c) majorizing the approximant at µk = 6. Here, = 1/5.

Figure 4: Ratio of L(θ̂) found by the MM algorithm to L(θ̂) found by the interior point method for various values of q, averaged over 100 repetitions. The problem used here is linear with n = 100 and p = 5.

Figure 5: log10 of mean FLOP count for 10 runs of both methods on a nonlinear problem for various values of q, p, and n. Solid dots represent MM runs; open circles represent interior point runs.

Figure 3: log10 of mean FLOP count for 10 runs of both methods on a linear problem for various values of q, p, and n. Solid dots represent MM runs; open circles represent interior point runs.

Citations

Journal Article•10.1198/0003130042836

A Tutorial on MM Algorithms

David R. Hunter, +1 more

- 01 Feb 2004

- The American Statistician

TL;DR: The principle behind MM algorithms is explained, some methods for constructing them are suggested, and some of their attractive features are discussed.

...read moreread less

2.1K

Journal Article•10.1080/10618600.2000.10474858

Optimization Transfer Using Surrogate Objective Functions

Kenneth Lange, +2 more

- 01 Mar 2000

- Journal of Computational and Graphical S...

TL;DR: Because optimization transfer algorithms often exhibit the slow convergence of EM algorithms, two methods of accelerating optimization transfer are discussed and evaluated in the context of specific problems.

...read moreread less

950

•Posted Content

Conformalized Quantile Regression

Yaniv Romano, +2 more

- 08 May 2019

- arXiv: Methodology

TL;DR: This paper proposes a new method that is fully adaptive to heteroscedasticity, which combines conformal prediction with classical quantile regression, inheriting the advantages of both.

...read moreread less

363

•Journal Article•10.3758/BF03196299

Quantile maximum likelihood estimation of response time distributions.

Andrew Heathcote, +2 more

- 01 Jun 2002

- Psychonomic Bulletin & Review

TL;DR: A robust new estimation technique is introduced and evaluated via a Monte Carlo study that fits distribution functions to grouped response time (RT) data, where the grouping is determined by sample quantiles, which is more efficient and less biased than the best alternative estimation technique.

...read moreread less

266

Journal Article•10.1109/TWC.2014.2352619

Downlink and Uplink Energy Minimization Through User Association and Beamforming in C-RAN

Shixin Luo, +2 more

- 01 Jan 2015

- IEEE Transactions on Wireless Communicat...

TL;DR: In this paper, the authors proposed a joint downlink (DL) and UL MU-AP association and beamforming design to coordinate interference in the C-RAN for energy minimization, a problem which is shown to be NP hard.

...read moreread less

265

...

Expand

References

Journal Article•10.1111/J.2517-6161.1977.TB01600.X

Maximum likelihood from incomplete data via the EM algorithm

Arthur P. Dempster, +2 more

- 01 Sep 1977

- Journal of the royal statistical society...

55.2K

Journal Article•10.1063/1.4822961

Numerical recipes

William H. Press, +1 more

- 01 Oct 1990

- Computers in Physics

TL;DR: This section discusses a particular type of low-pass filter, well-adapted for data smoothing, and termed variously Savitzky-Golay, least-squares, or DISPO (Digital Smoothing Polynomial) filters.

...read moreread less

19K

•Book

Iterative Solution of Nonlinear Equations in Several Variables

J.M. Ortega, +1 more

- 01 Jun 1970

TL;DR: In this article, the authors present a list of basic reference books for convergence of Minimization Methods in linear algebra and linear algebra with a focus on convergence under partial ordering.

...read moreread less

7.9K

•Book

Robust Regression and Outlier Detection

Peter J. Rousseeuw, +1 more

- 01 Jan 1987

TL;DR: This paper presents the results of a two-year study of the statistical treatment of outliers in the context of one-Dimensional Location and its applications to discrete-time reinforcement learning.

...read moreread less

•Book

The EM algorithm and extensions

Geoffrey J. McLachlan, +1 more

- 15 Nov 1996

TL;DR: The EM Algorithm and Extensions describes the formulation of the EM algorithm, details its methodology, discusses its implementation, and illustrates applications in many statistical contexts, opening the door to the tremendous potential of this remarkably versatile statistical tool.

...read moreread less

6.7K

...

Expand

Quantile Regression via an MM Algorithm

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What are the contributions mentioned in the paper "Quantile regression via an mm algorithm" ?

2. What is the best-known example of an MM algorithm?

3. What are the main strengths of the MM algorithm presented in this paper?

4. What is the approach to the minimization problem?

5. What is the kth iterate of this classical algorithm?

6. Why is the objective function L() difficult to minimize?

7. What is the technical conditionlim for q?

8. What is the criterion used for the MM algorithm?

9. What is the argument used in the proof of Proposition 4?

10. What is the q quantile of an integrable random variable?

Figures

Citations

A Tutorial on MM Algorithms

Optimization Transfer Using Surrogate Objective Functions

Conformalized Quantile Regression

Quantile maximum likelihood estimation of response time distributions.

Downlink and Uplink Energy Minimization Through User Association and Beamforming in C-RAN

References

Maximum likelihood from incomplete data via the EM algorithm

Numerical recipes

Iterative Solution of Nonlinear Equations in Several Variables

Robust Regression and Outlier Detection

The EM algorithm and extensions

Related Papers (5)

Regression Shrinkage and Selection via the Lasso

Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

A Tutorial on MM Algorithms

Learning the parts of objects by non-negative matrix factorization

Quantile Regression: Name Index