Abstract: Classification methods from statistical pattern recognition, neural nets, and machine learning were applied to four real-world data sets. Each of these data sets has been previously analyzed and reported in the statistical, medical, or machine learning literature. The data sets are characterized by statisucal uncertainty; there is no completely accurate solution to these problems. Training and testing or resampling techniques are used to estimate the true error rates of the classification methods. Detailed attention is given to the analysis of performance of the neural nets using back propagation. For these problems, which have relatively few hypotheses and features, the machine learning procedures for rule induction or tree induction clearly performed best.

Abstract: Although general network learning rules are of undeniable interest, it is generally agreed that successful accounts of learning must incorporate domain-specific, a priori knowledge. Such knowledge might be used, for example, to determine the structure of a network or its initial weights. The author discusses a third possibility in which domain-specific knowledge is incorporated directly in a network learning rule via a set of constraints on activations. The approach uses the notion of a forward model to give constraints a domain-specific interpretation. This approach is demonstrated with several examples from the domain of motor learning. >

Abstract: The structural stability of real-time unsupervised learning in feedback dynamical systems is demonstrated with the Ito-Stratonovich stochastic calculus. Structural stability allows globally stable feedback systems to be perturbed without changing their qualitative equilibrium behavior. This increases the reliability and biological plausibility of large-scale hardware implementations of such networks. These structurally stable dynamical systems are called random adaptive bidirectional associative memory (RABAM) models. RABAM models include several popular nonadaptive and adaptive feedback models, such as the Hopfield circuit and the ART-2 model. A new hybrid unsupervised-learning law, called the differential competitive law, which uses the signal velocity as a local unsupervised reinforcement mechanism, is introduced, and its coding and stability behavior in feedforward and feedback networks is studied. >

Abstract: Several published reports show that instancebased learning algorithms yield high classification accuracies and have low storage requirements during supervised learning applications. However, these learning algorithms are highly sensitive to noisy training instances. This paper describes a simple extension of instancebased learning algorithms for detecting and removing noisy instances from concept descriptions. This extension requires evidence that saved instances be significantly good classifiers before it allows them to be used for subsequent classification tasks. We show that this extension's performance degrades more slowly in the presence of noise, improves classification accuracies, and further reduces storage requirements in several artificial and real-world database applications.

Abstract: A theory and prototype of a neural controller called INFANT, which learns sensory-motor coordination from its own experience, is presented. INFANT adapts to unforeseen changes in the geometry of the physical motor system and to the location, orientation, shape, and size of objects. It can learn to accurately grasp an elongated object without any information about the geometry of the physical sensory-motor system. INFANT relies on the self-consistency between sensory and motor signals to achieve unsupervised learning. It is designed to be generalized for coordinating any number of sensory inputs with limbs of any number of joints. INFANT is implemented with an image processor, stereo cameras, and a five-degree-of-freedom robot arm. After learning, its average position accuracy is within 3% of the length of the arm, and its orientation accuracy is within 60 degrees in solid angle. >

Abstract: This paper considers the problem of learning classification rules from data in the context of knowledge acquisition. Bayesian theory provides a framework for both designing learning algorithms and for approaching specific learning applications, for instance, in the selection and tuning of learning tools. Experiments are reported demonstrating how this can be done using two well known learning approaches: simple Bayes classifiers and decision trees.

Abstract: In associative reinforcement learning, an environment generates input vectors, a learning system generates possible output vectors, and a reinforcement function computes feedback signals from the input-output pairs. The task is to discover and remember input-output pairs that generate rewards. Especially difficult cases occur when rewards are rare, since the expected time for any algorithm can grow exponentially with the size of the problem. Nonetheless, if a reinforcement function possesses regularities, and a learning algorithm exploits them, learning time can be reduced below that of non-generalizing algorithms. This paper describes a neural network algorithm called complementary reinforcement back-propagation (CRBP), and reports simulation results on problems designed to offer differing opportunities for generalization.

Abstract: A method is given for unsupervised segmentation and classification of 1D and 2D signals. The method is based on a self-organizing map of ”neural” units, like Kohonen’s feature map, where every unit represents an AR model with its reference vector. The map will self-organize during an unsupervised learning phase. Several training segments of the signals are presented to the map, and each unit will learn to model different parts of the signals. The results indicate that the self-organizing AR map can learn to distinguish textures from images with unsupervised learning, which makes it suitable for segmentation of an image into different texture classes.

Abstract: A theory and the prototype of a neural controller called INFANT that learns sensory-motor coordination from its own experience are presented. INFANT adapts unforeseen changes in the geometry of the physical motor system and to the location, orientation, shape, and size of objects. It can learn to accurately grasp an elongated object without any information about the geometry of the physical sensory-motor system. This new neural controller relies on the self-consistency between sensory and motor signals to achieve unsupervised learning. It is designed to be generalized for coordinating any number of sensory inputs with limbs of any number of joints. INFANT is implemented with an image processor, stereo cameras, and a 5 degrees-of-freedom robot arm. Its average grasping accuracy after learning is 3% of the arm's length in position and 6 degrees in orientation. >

Abstract: In many applications in mobile robotics, it is important for a robot to explore its environment in order to construct a representation of space useful for guiding movement. We refer to such a representation as a map, and the process of constructing a map from a set of measurements as map learning. In this paper, we develop a framework for describing map-learning problems in which the measurements taken by the robot are subject to known errors. We investigate two approaches to learning maps under such conditions: one based on Valiant's probably approximately correct learning model, and a second based on Rivest Sz Sloan's reliable and probably nearly almost always useful learning model. Both methods deal with the problem of accumulated error in combining local measurements to make global inferences. In the first approach, the effects of accumulated error are eliminated by the use of reliable and probably useful methods for discerning the local properties of space. In the second, the effects of accumulated error are reduced to acceptable levels by repeated exploration of the area to be learned. Finally, we suggest some insights into why certain existing techniques for map learning perform as well as they do.

Abstract: Pattern recognition methods, particularly the ‘unsupervised learning’ techniques, are well suited for the preliminary analysis of the large data sets produced by computer chemistry The use of linear and non-linear display methods for such exploratory analysis are exemplified with the aid of two data sets of biologically active molecules Advantages and disadvantages of these techniques are discussed

Abstract: In this paper, we consider the problem of correlational learning and present efficient algorithms to determine correlated objects.

Abstract: This paper describes an approach to combining empirical and analytical learning using incremental version-space merging (Hirsh, 1989). The basic idea is to use analytical learning to generalize training data before doing empirical learning. The combination operates like empirical learning given no knowledge, but can utilize knowledge when provided, and thus exhibits behavior along a spectrum from knowledge-poor to knowledge-rich learning.

Abstract: A method of learning structural models of 2D shape from real data is described and demonstrated. These models can be used to classify nonrigid shapes, even if they are partially occluded, and to label their parts. The representation of a single shape is a layered graph whose vertices correspond to n-ary relations. A class of shapes is represented as a probability model whose outcome is a graph. The method is based on two types of learning: unsupervised learning used to discover relations, and supervised learning used to build class models. The class models are constructed incrementally, by matching and merging graphs representing shape instances. This process uses a fast graph-matching heuristic which seeks a simplest representation of a graph. An important feature is the self-generation of symbolic primitives by an unsupervised learning process. This feature makes it possible to apply the system to any set of shape data without adjustments, while other methods might require the user to provide a different set of primitives for each case. >

Abstract: We investigate the properties of feedforward neural networks trained with Hebbian learning algorithms. A new unsupervised algorithm is proposed which produces statistically uncorrelated outputs. The algorithm causes the weights of the network to converge to the eigenvectors of the input correlation with largest eigenvalues. The algorithm is closely related to the technique of Self-supervised Backpropagation, as well as other algorithms for unsupervised learning. Applications of the algorithm to texture processing, image coding, and stereo depth edge detection are given. We show that the algorithm can lead to the development of filters qualitatively similar to those found in primate visual cortex.

Abstract: The development of a system to detect online multichannel epileptiform spikes is described. Three main topics are discussed. The first is the preprocessing procedure used on the raw data prior to their presentation to the neural network. Issues reviewed include tradeoffs between preprocessing and system complexity. The second is the development of CaseNet, a neural network development tool used to graphically specify a network architecture from which executable code is generated automatically. Areas discussed include selection of the network architecture, such as choices between supervised and unsupervised learning schemes. The third concerns the interim results of the analysis of single- and four-channel electroencephalogram (EEG) data. The relationship of the spike detection effort to a similar one for seizure detection is also outlined. >

Abstract: The authors discuss some problems in learning networks. They propose a new learning procedure, CALM (Categorizing And Learning Module). CALM uses pairs of excitatory representation nodes and inhibitory veto nodes, bound together in a modular structure with an arousal node. Learning in the module is enhanced by a nonspecific external node connected to the arousal node. The system is capable of both supervised and unsupervised learning and can both discriminate and generalize across similar patterns. A system constructed out of several CALM modules is shown to learn the XOR relationship with supervised and unsupervised presentation. It also models list recall and word completion memory tasks and can learn, unsupervised, handwritten digits and recognize them with unknown authors. >

Abstract: An adaptive pattern recognition methodology for online monitoring and diagnosis of power system operating conditions has been developed. It is implemented on highly parallel distributed architectures of the functional-link-net (FLN) type. The flat structure of the FLN allows the tasks of unsupervised learning, supervised learning, and associative recall to be carried out without intervention in network and data structures. The proposed methodology is capable of processing large bodies of information gathered by the data acquisition system in real time. It enhances the performance of the energy management system and effectively reduces the operator's response time. The real-time monitoring and diagnosis facility can quickly detect and identify abnormal operating conditions. The main features of the system are described. >

Abstract: In this paper we study the role of supervised and unsupervised neural learning schemes in the adaptive control of nonlinear dynamic systems. We suggest and demonstrate that the teacher's knowledge in the supervised learning mode includes a-priori plant sturctural knowledge which may be employed in the design of exploratory schedules during learning that results in an unsupervised learning scheme. We further demonstrate that neurocontrollers may realize both linear and nonlinear control laws that are given explicitly in an automated teacher or implicitly through a human operator and that their robustness may be superior to that of a model based controller. Examples of both learning schemes are provided in the adaptive control of robot manipulators and a cart-pole system.

Abstract: Any nonassociative reinforcement learning algorithm can be viewed as a method for performing function optimization through (possibly noise-corrupted) sampling of function values. A description is given of the results of simulations in which the optima of several deterministic functions studied by D.H. Ackley (Ph.D. Diss., Carnegie-Mellon Univ., 1987) were sought using variants of REINFORCE algorithms. Results obtained for certain of these algorithms compare favorably to the best results found by Ackley. >

Abstract: The paper suggests a statistical framework for the parameter estimation problem associated with unsupervised learning in a neural network, leading to an exploratory projection pursuit network that performs feature extraction, or dimensionality reduction.

Abstract: The effect of data character on empirical concept learning has typically been studied using data from real domains. This presents problems because such data is often limited and uncontrollable. The authors present a more complex approach. They examine typical problems in an effort to characterize concepts more completely. They than utilize the characterization to generate data artificially. From this controlled data, they measure learning performance (speed and accuracy) as a precise function of several data characteristics. The authors' experiments lead to some novel conclusions: a useful starting point to clarify data character is the definition of the term concept, which is effectively a function over instance space; characterizing a concept as a function allows the mimicking of natural data and the control of the generation of artificial data for extensive experimentation; data characteristics are numerous and easy to overlook; and, compared with significant design factors of learning algorithms, certain data characteristics are highly significant. >

Abstract: A learning control approach called refinement, in which a fixed controller is first designed using analytic design tools is explored. This controller's performance is refined by a secondary learning controller, which is a reinforcement learning-based connectionist network. The issue is the representation of the input space of the refinement learning controller. In previous work, the input space was quantized into fixed boxes and each box became a control situation for the learning controller. The drawback was that the learning control designer had to know how to quantize the space. An approach to eliminating the quantization of the input space is described. The new input space representation consists of functions that act as receptive fields and have the shape of multivariate Gaussian probability density functions; they are the first layer in the learning network. Experiments used a tracking control problem with an additive nonlinearity. The learning controller adds an appropriate control signal on the basis of a given evaluation function, in order to improve the fixed controller's ability to track a reference signal. >

Abstract: Neural networks are being used to analyze electroencephalogram (EEG) signals for the detection of epileptiform spikes. A review is presented of the design considerations involved in implementing a real-time spike detection system. Issues addressed are generally in two areas. The first is the characterization of the source data. For example, decisions must be made relative to data rates, the number of data channels and whether to use raw data, or preprocessed data in the form of spike parameters. The second is the selection of the neural network architecture and the specific implementation of that architecture. For example, choices must be made between supervised and unsupervised learning schemes, and among the many available network learning algorithms. A discussion is presented of interim results in an EEG spike detection project, the goal of which is to provide real-time spike detection capability for a multibed epilepsy monitoring unit. >

Abstract: Knowledge acquisition systems with a model-driven learning mechanism require the representation of that model in the system. The model which guides the learning mechanism must be distinguished from the knowledge (domain model) which is to be learned with the learning mechanism; only the former is the concern of this paper. If the model for guiding the learning mechanism is to be enlarged and improved while working with such a system, the acquisition and representation of new parts of this model must be supported. In addition to the insertion of new parts into the existing model, it is very important to consider redundancy, integrity and completion, because the quality of the model influences the quality of the learning capabilities of the knowledge acquisition system.