Showing papers on "Weighted Majority Algorithm published in 1999"
29 Nov 1999
TL;DR: This work describes a new incremental algorithm for training linear threshold functions: the Relaxed Online Maximum Margin Algorithm, or ROMMA, and proves a mistake bound for ROMMA that is the same as that proved for the perceptron algorithm.
Abstract: We describe a new incremental algorithm for training linear threshold functions: the Relaxed Online Maximum Margin Algorithm, or ROMMA. ROMMA can be viewed as an approximation to the algorithm that repeatedly chooses the hyperplane that classifies previously seen examples correctly with the maximum margin. It is known that such a maximum-margin hypothesis can be computed by minimizing the length of the weight vector subject to a number of linear constraints. ROMMA works by maintaining a relatively simple relaxation of these constraints that can be efficiently updated. We prove a mistake bound for ROMMA that is the same as that proved for the perceptron algorithm. Our analysis implies that the maximum-margin algorithm also satisfies this mistake bounds this is the first worst-case performance guarantee for this algorithm. We describe some experiments using ROMMA and a variant that updates its hypothesis more aggressively as batch algorithms to recognize handwritten digits. The computational complexity and simplicity of these algorithms is similar to that of perceptron algorithm, but their generalization is much better. We show that a batch algorithm based on aggressive ROMMA converges to the fixed threshold SVM hypothesis.
273 citations
TL;DR: The ILA-2 rule induction algorithm is described, which is the improved version of a novel inductive learning algorithm ILA, and how the algorithm is improved using a new evaluation metric that handles uncertainty in the data.
Abstract: In this paper we describe the ILA-2 rule induction algorithm, which is the improved version of a novel inductive learning algorithm ILA . We first outline the basic algorithm ILA, and then present how the algorithm is improved using a new evaluation metric that handles uncertainty in the data. By using a new soft computing metric, users can reflect their preferences through a penalty factor to control the performance of the algorithm. Inductive learning algorithm has also a faster pass criteria feature which reduces the processing time without sacrificing much from the accuracy that is not available in basic ILA. We experimentally show that the performance of ILA-2 is comparable to that of well-known inductive learning algorithms, namely, CN2, OC1, ID3, and C4.5.
35 citations
Proceedings Article•
30 Jul 1999TL;DR: Two techniques are described that significantly improve the running time of several standard machine-learning algorithms when data is sparse and perform the E-step of the EM algorithm (i.e., inference) when applied to a naive-Bayes clustering model.
Abstract: We describe two techniques that significantly improve the running time of several standard machine-learning algorithms when data is sparse. The first technique is an algorithm that efficiently extracts one-way and two-way counts-either real or expected-from discrete data. Extracting such counts is a fundamental step in learning algorithms for constructing a variety of models including decision trees, decision graphs, Bayesian networks, and naive-Bayes clustering models. The second technique is an algorithm that efficiently performs the E-step of the EM algorithm (i.e., inference) when applied to a naive-Bayes clustering model. Using real-world data sets, we demonstrate a dramatic decrease in running time for algorithms that incorporate these techniques.
19 citations
31 Aug 1999
TL;DR: An improved learning algorithm for rule refinement in neuro-fuzzy modeling is proposed which is mainly based on a well-known technique, i.e., singular value decomposition (SVD).
Abstract: We propose an improved learning algorithm for rule refinement in neuro-fuzzy modeling. This algorithm is mainly based on a well-known technique, i.e., singular value decomposition (SVD). By using the method of SVD, the learning algorithm can converge quickly. Besides, the reasoning operator adopted in our algorithm is a compensatory fuzzy operator which has the advantage of being more adaptive and effective. Experimental results show that the proposed algorithm converges quickly and the obtained fuzzy rules are more precise.
15 citations
10 Jul 1999
TL;DR: A block formulation of the Levenberg-Marquardt algorithm to train feedforward MLPs is designed to track time-varying nonlinear functions and shows performance that is superior to the performance of existing algorithms like the global extended Kalman filter algorithm with state noise in the system equations.
Abstract: A block formulation of the Levenberg-Marquardt algorithm to train feedforward MLPs is designed to track time-varying nonlinear functions. The resulting algorithm is called the block Levenberg-Marquardt algorithm. There are two varieties of the algorithm: the overlapping and the non-overlapping block Levenberg-Marquardt. The two algorithms are developed in terms of a block presentation of the input/output training set. The tracking problem can be viewed as one of solving a sequence of nonlinear identification problems. With the persistent excitation and slowly-varying system conditions satisfied, the Levenberg-Marquardt algorithm can be shown to have a uniform rate of convergence over the entire sequence of problems. The block Levenberg-Marquardt algorithms are tested on a nonlinear time-varying function tracking problem. The algorithms show performance that is superior to the performance of existing algorithms like the global extended Kalman filter algorithm with state noise in the system equations.
14 citations
1 Dec 1999
TL;DR: This work presents an efficient method for statistical parameter learning of a certain class of symbolic-statistical models including hidden Markov models (HMMs), and shows that, given appropriate data structure, Baum-Welch algorithm can be simulated by the authors' graph-based EM algorithm.
Abstract: We present an efficient method for statistical parameter learning of a certain class of symbolic-statistical models (called PRISM programs) including hidden Markov models (HMMs). To learn the parameters, we adopt the EM algorithm, an iterative method for maximum likelihood estimation. For the efficient parameter learning, we first introduce a specialized data structure for explanations for each observation, and then apply a graph-based EM algorithm. The algorithm can be seen as a generalization of Baum-Welch algorithm, an EM algorithm specialized for HMMs. We show that, given appropriate data structure, Baum-Welch algorithm can be simulated by our graph-based EM algorithm.
8 citations
1 Oct 1999
TL;DR: The proposed learning algorithm is a generalization of Baba's GAE algorithm and has applications in solving, in a parallel manner, multi-objective optimization problems in which each objective function is disturbed by noise.
Abstract: Learning algorithms for an automaton operating in a multiteacher environment are considered. These algorithms are classified based on the number of actions given as inputs to the environments and the number of responses (outputs) obtained from the environments. In this paper, we present a general class of learning algorithm for multi-input multi-output (MIMO) models. We show that the proposed learning algorithm is absolutely expedient and /spl epsiv/-optimal in the sense of average penalty. The proposed learning algorithm is a generalization of Baba's GAE algorithm and has applications in solving, in a parallel manner, multi-objective optimization problems in which each objective function is disturbed by noise.
8 citations
1 Jan 1999
TL;DR: A weighted instance-based algorithm that combines the K-nearest neighbour technique and a distance metric which provides selective spreading of learning updates on the experience space generates a good action policy for a simulated guidance robot.
Abstract: We report preliminary results on a weighted instance-based algorithm for the problem of autonomous robot learning. The algorithm combines the K-nearest neighbour technique and a distance metric which provides selective spreading of learning updates on the experience space, with the aim of minimizing the problem of partial state observability produced by local sensor readings and insufficient global information. Results show that the algorithm generated a good action policy for a simulated guidance robot with two basic behaviours (preprogrammed obstacle avoidance and learned target approximation).
6 Jul 1999
TL;DR: A new boosting algorithm is proposed that is an adaptive version of the boost by majority algorithm and combines bounded goals of the boosted algorithm with the adaptivity of AdaBoost.
Abstract: We propose a new boosting algorithm. This boosting algorithm is an adaptive version of the boost by majority algorithm and combines bounded goals of the boost by majority algorithm with the adaptivity of AdaBoost.
17 Dec 1999
TL;DR: Two improved list-ranking algorithms are presented, one of which leads to an optimal PRAM algorithm, but was designed with application on a real parallel machine in mind, and the other is simpler than earlier algorithms, and in a range of problem sizes.
Abstract: Two improved list-ranking algorithms are presented. The “peeling-off” algorithm leads to an optimal PRAM algorithm, but was designed with application on a real parallel machine in mind. It is simpler than earlier algorithms, and in a range of problem sizes, where previously several algorithms where required for the best performance, now this single algorithm suffices. If the problem size is much larger than the number of available processors, then the “sparse-ruling-sets” algorithm is even better. In previous versions this algorithm had very restricted practical application because of the large number of communication rounds it was performing. This weakness is overcome by adding two new ideas, each of which reduces the number of communication rounds by a factor of two.