Open AccessProceedings Article
Margin distribution and learning algorithms
Ashutosh Garg,Dan Roth +1 more
- 21 Aug 2003
- pp 210-217
TL;DR: This paper enhances earlier theoretical results and derives a practical data-dependent complexity measure for learning, which is a function of the observed margin distribution of the data, and can be used, as it is shown, as a model selection criterion.
read more
Abstract: Recent theoretical results have shown that improved bounds on generalization error of classifiers can be obtained by explicitly taking the observed margin distribution of the training data into account. Currently, algorithms used in practice do not make use of the margin distribution and are driven by optimization with respect to the points that are closest to the hyperplane.
This paper enhances earlier theoretical results and derives a practical data-dependent complexity measure for learning. The new complexity measure is a function of the observed margin distribution of the data, and can be used, as we show, as a model selection criterion. We then present the Margin Distribution Optimization (MDO) learning algorithm, that directly optimizes this complexity measure. Empirical evaluation of MDO demonstrates that it consistently outperforms SVM.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
The Extreme Value Machine
TL;DR: The Extreme Value Machine (EVM) is a novel, theoretically sound classifier that has a well-grounded interpretation derived from statistical Extreme Value Theory (EVT), and is the first classifier to be able to perform nonlinear kernel-free variable bandwidth incremental learning.
377
Large margin distribution machine
Teng Zhang,Zhi-Hua Zhou +1 more
- 24 Aug 2014
TL;DR: The Large margin distribution machine (LDM) as mentioned in this paper is a general learning approach which can be used in any place where SVM can be applied, and its superiority is verified both theoretically and empirically in this paper.
118
•Journal Article
Noise Tolerant Variants of the Perceptron Algorithm
Roni Khardon,Gabriel Wachman +1 more
TL;DR: The perceptron with margin is an effective method for tolerating noise and stabilizing the algorithm and highlights the difficulty with automatic parameter selection which is required with some of these variants of the Perceptron algorithm.
•Posted Content
Large Margin Distribution Machine
Teng Zhang,Zhi-Hua Zhou +1 more
TL;DR: The Large margin Distribution Machine (LDM), which tries to achieve a better generalization performance by optimizing the margin distribution, is proposed and its superiority is verified both theoretically and empirically in this paper.
69
Large Margin Distribution Learning
Zhi-Hua Zhou
- 06 Oct 2014
TL;DR: Inspired by the recent theoretical results, this work advocates the large margin distribution learning, a promising research direction that has exhibited superiority in algorithm designs to traditional large margin learning.
References
Statistical learning theory
Vladimir Vapnik
- 01 Jan 1998
TL;DR: Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more.
30.4K
•Journal Article
Extensions of Lipschitz mappings into Hilbert space
Abstract: (Here ll&lltip is the Lipschitz constant of the function g.) A classical result of Kirszbraun's [14, p. 48] states that L(t2, n) = 1 for all n, but it is easy to see that L(X, n) ~ ~ as n ~ ~ for many metric spaces X. Marcus and Pisier [10] initiated the study of L(X, n) for X = Lp. (For brevity, we will use hereafter the notation L(p, n) for L(Lp(O,l), n).) They prove that for each 1 < p < 2 there is a constant C(p) so that for n = 2, 3, 4, , , ,
3.4K
•Proceedings Article
Boosting the margin: A new explanation for the effectiveness of voting methods
Robert E. Schapire,Yoav Freund,Peter Barlett,Wee Sun Lee +3 more
- 08 Jul 1997
TL;DR: In this paper, the authors show that the test error of the generated classifier usually does not increase as its size becomes very large, and often is observed to decrease even after the training error reaches zero.
•Book
Neural Network Learning: Theoretical Foundations
Martin Anthony,Peter L. Bartlett +1 more
- 01 Nov 1999
TL;DR: The authors explain the role of scale-sensitive versions of the Vapnik Chervonenkis dimension in large margin classification, and in real prediction, and discuss the computational complexity of neural network learning.
2K
Related Papers (5)
Vladimir Vapnik
- 01 Jan 1998
Wei Gao,Zhi-Hua Zhou +1 more
Teng Zhang,Zhi-Hua Zhou +1 more
- 24 Aug 2014