Showing papers on "Radial basis function network published in 1994"
TL;DR: This analysis makes clear the strong similarity between linear neural networks and the general linear model developed by statisticians.
Abstract: Neural networks are composed of basic units somewhat analogous to neurons. These units are linked to each other by connections whose strength is modifiable as a result of a learning process or algorithm. Each of these units integrates independently (in paral lel) the information provided by its synapses in order to evaluate its state of activation. The unit response is then a linear or nonlinear function of its activation. Linear algebra concepts are used, in general, to analyze linear units, with eigenvectors and eigenvalues being the core concepts involved. This analysis makes clear the strong similarity between linear neural networks and the general linear model developed by statisticians. The linear models presented here are the perceptron and the linear associator. The behavior of nonlinear networks can be described within the framework of optimization and approximation techniques with dynamical systems (e.g., like those used to model spin glasses). One of the main notions used with nonlinear unit networks is the notion of attractor. When the task of the network is to associate a response with some specific input patterns, the most popular nonlinear technique consists of using hidden layers of neurons trained with back-propagation of error. The nonlinear models presented are the Hopfield network, the Boltzmann machine, the back-propagation network and the radial basis function network.
163 citations
TL;DR: This work shows that the adaptive radial basis function network attains near optimal performance and is robust in realistic communication environments.
Abstract: Adaptive methods for performing multiuser demodulation in a direct-sequence spread-spectrum multiple-access (DS/SSMA) communication environment are investigated. In this scenario, the noise is characterized as being the sum of the interfering users' signals and additive Gaussian noise. The optimal receiver for DS/SSMA systems has a complexity that is exponential in the number of users. This prohibitive complexity has spawned the area of research on suboptimal receivers with moderate complexity. Adaptive algorithms for detection allow for reception when the communication environment is either unknown or changing. Motivated by previous work with radial basis functions (RBF's) for performing equalization, RBF networks that operate with knowledge of only a subset of the system parameters are studied. Although this form of detection has been previously studied (group detection) when the system parameters are known, in this work, neural network techniques are employed to adaptively determine unknown system parameters. This approach is further bolstered by the fact that the optimal detector in the synchronous case can be implemented by a RBF network when all of the system parameters are known. The RBF network's performance (with estimated parameters) is compared with the optimal synchronous detector, the decorrelating detector and the single layer perceptron detector. Clustering techniques and adaptive least mean squares methods are investigated to determine the unknown system parameters. This work shows that the adaptive radial basis function network attains near optimal performance and is robust in realistic communication environments. >
160 citations
TL;DR: It is shown that the optimal Bayesian equaliser is structurally equivalent to the complex radial basis function network, and this intimate connection is exploited to develop fast training algorithms for implementing a Bayesianequaliser based on the latter.
148 citations
TL;DR: A complex radial basis function network has complex centres and connection weights, but the nonlinearity of its hidden nodes remains a real-valued function, and it provides a powerful tool for nonlinear signal processing involving complex signals.
132 citations
11 Nov 1994
Abstract: A radial basis function network architecture is developed that learns the correlation of facial feature motion patterns and human emotions. We describe a hierarchical approach which at the highest level identifies emotions, at the mid level determines motion of facial features, and at the low level recovers motion directions. Individual emotion networks were trained to recognize the 'smile' and 'surprise' emotions. Each emotion network was trained by viewing a set of sequences of one emotion for many subjects. The trained neural network was then tested for retention, extrapolation and rejection ability. Success rates were about 88% for retention, 73% for extrapolation, and 79% for rejection. >
110 citations
TL;DR: This paper considers a least-squares approach to function approximation and generalization and shows that better generalization will occur if the error criterion used in training the generalizer is modified by the addition of a specific regularization term.
Abstract: This paper considers a least-squares approach to function approximation and generalization. The particular problem addressed is one in which the training data are noiseless and the requirement is to define a mapping that approximates the data and that generalizes to situations in which data samples are corrupted by noise in the input variables. The least-squares approach produces a generalizer that has the form of a radial basis function network for a finite number of training samples. The finite sample approximation is valid provided that the perturbations due to noise on the expected operating conditions are large compared to the sample spacing in the data space. In the other extreme of small noise perturbations, a particular parametric form must be assumed for the generalizer. It is shown that better generalization will occur if the error criterion used in training the generalizer is modified by the addition of a specific regularization term. This is illustrated by an approximator that has a feedforward architecture and is applied to the problem of point-source location using the outputs of an array of receivers in the focal-plane of a lens. >
97 citations
1 Jan 1994
TL;DR: This paper will show how an approach based on regularization theory leads to develop a family of approximation techniques, including Radial Basis Functions, and some tensor product and additive splines, and how this fairly classical approach has to be extended in order to cope with special features of the problem of learning of examples.
Abstract: This paper consists of two parts. In the first part we consider the problem of learning from examples in the setting of the theory of the approximation of multivariate functions from sparse data. We first will show how an approach based on regularization theory leads to develop a family of approximation techniques, including Radial Basis Functions, and some tensor product and additive splines. Then we will show how this fairly classical approach has to be extended in order to cope with special features of the problem of learning of examples, such as high dimensionality and strong anisotropics. Furthermore, the same extension that leads from Radial Basis Functions (RBF) to Hyper Basis Functions (HBF) also leads from additive models to ridge approximation models, such as some forms of Projection Pursuit Regression.
95 citations
27 Jun 1994
TL;DR: The results suggest an equivalent performance of TDR BF and TDNN presented in Waibel et al. (1989), but TDRBF require much less training time to reach a good performance and in addition have a clear indication when the minimum error is reached, therefore no danger of overtraining exists.
Abstract: This paper presents the time delay radial basis function network (TDRBF) for recognition of phonemes. The TDRBF combines features from time delay neural networks (TDNN) and radial basis functions (RBF). The ability to detect acoustic features and their temporal relationship independent of position in time is inherited from TDNN. The use of RBFs leads to shorter training times and less parameters to adjust, which makes it easier to apply TDRBF to new tasks. The recognition of three phonemes with about 750 training and testing tokens each was chosen as an evaluation task. The results suggest an equivalent performance of TDRBF and TDNN presented in Waibel et al. (1989), but TDRBF require much less training time to reach a good performance and in addition have a clear indication when the minimum error is reached, therefore no danger of overtraining exists. >
69 citations
TL;DR: The authors propose the use of a radial basis function (RBF) network for direction-of-arrival (DOA) estimation and results show that the new technique has a better performance in terms of estimation errors than the standard MUSIC algorithm.
Abstract: The authors propose the use of a radial basis function (RBF) network for direction-of-arrival (DOA) estimation. The RBF network is used to approximate the functional relationship between sensor outputs and the direction of arrivals. Simulation results show that the new technique has a better performance in terms of estimation errors than the standard MUSIC algorithm. >
53 citations
1 Dec 1994
Abstract: Adaptive learning dynamics of the radial basis function network (RBFN) are compared with a scale-based clustering technique and a relationship between the two is pointed out. Using this link, it is shown how scale-based clustering can be done using the RBFN, with the radial basis function (RBF) width as the scale parameter. The technique suggests the "right" scale at which the given data set must be clustered and obviates the need for knowing the number of clusters beforehand. We show how this method solves the problem of determining the number of RBF units and the widths required to get a good network solution. >
45 citations
TL;DR: It is shown that the successive approximation radial basis network presented here is a useful tool for nonlinear modeling and prediction.
TL;DR: This paper compares the performance of two neural network models for fault diagnosis of power systems with reference to generalization, training time and number of training patterns needed for each model.
2 Oct 1994
TL;DR: By applying the RBFN-based sliding mode controller to control a nonlinear unstable inverted pendulum system, the simulation results show the expected approximation sliding property was occurred, and the dynamic behavior of the control system can be determined by the sliding surface.
Abstract: A sliding mode controller (SMC) design method based on radial basis function network (RBFN) is proposed in this paper. Similar to the multilayer perceptron, the RBFN also known to be a good universal approximator. In this work, the weights of the RBFN are changed according to some adaptive algorithms for the purpose of controlling the system state to hit a user-defined sliding surface and then slide along it. The initial weights of the RBFN can be set to small random numbers, and then online tuned automatically, no supervised learning procedures are needed. By applying the RBFN-based sliding mode controller to control a nonlinear unstable inverted pendulum system, the simulation results show the expected approximation sliding property was occurred, and the dynamic behavior of the control system can be determined by the sliding surface. >
14 Dec 1994
TL;DR: The paper proposes an extension of the Levenberg-Marquardt algorithm for numerical solution of the formulated problem, which approximates the nonlinear system by an expansion into a series of the parameter vector functions, affine in the input vector.
Abstract: This paper considers a parametric nonlinear least square (NLS) optimization problem. In extension of a classical NLS problem statement, it is assumed that the nonlinear optimized system depends on two arguments: an input vector and a parameter vector. The input vector can be modified to optimize the system, while the parameter vector changes from one optimization iteration to another and is not controlled. The optimization process goal is to find a dependence of the optimal input vector on the parameter vector, where the optimal input vector minimizes a quadratic performance index. The paper proposes an extension of the Levenberg-Marquardt algorithm for numerical solution of the formulated problem. The proposed algorithm approximates the nonlinear system by an expansion into a series of the parameter vector functions, affine in the input vector. In particular, a radial basis function network expansion is considered. The convergence proof for the algorithm is presented. >
1 Jan 1994
TL;DR: To achieve fast convergence, the learning algorithms were investigated to find fast and efficient procedures to automatically extract fuzzy rules and membership functions from high dimensional data.
Abstract: We summarize our results from investigating different learning and classification algorithms for basis function limited neural networks To achieve fast convergence we used RCE type learning procedures that have been modified for our applications and to enable simple hardware implementability The used radial and cubic basis functions are a signum type function, a ramp function and a gaussian function We investigated the learning algorithms to find fast and efficient procedures to automatically extract fuzzy rules and membership functions from high dimensional data which is topic of another paper >
TL;DR: It is proved that the distribution of centroids of the RBF network ap proaches node density of the Self-Organizing Feature Map as a limit, suggesting a deeper connection between the fundamental learning paradigms, namely supervised and unsupervised learning.
Abstract: The learning dynamics of a Radial Basis Function (RBF) network is shown to be related to the Learning Vector Quantization algorithm. Based on this similarity, a hybrid training scheme for the RBF network is proposed. The resulting Rapid Kernel Classifier is evaluated using a 6-class radar data set. Considerable speedup in training is obtained with this new scheme. Also, for the one-dimensional case, we prove that the distribution of centroids of the RBF network ap proaches node density of the Self-Organizing Feature Map as a limit. This result suggests a deeper connection between the fundamental learning paradigms, namely supervised and unsupervised learning.
29 Jun 1994
TL;DR: This paper considers radial basis function (RBF) network approximation of a multivariate nonlinear mapping as a linear parametric regression problem as well as formulation and proof of PE conditions on the input variables.
Abstract: We consider radial basis function (RBF) network approximation of a multivariate nonlinear mapping as a linear parametric regression problem. Linear recursive identification algorithms applied to this problem are known to converge, provided the regressor vector sequence has the persistency of excitation (PE) property. The contribution of this paper is formulation and proof of PE conditions on the input variables. In the RBF network identification, the regressor vector is a nonlinear function of these input variables. According to the formulated condition, the inputs provide PE, if they belong to domains around the network node centers.
TL;DR: In this paper, the authors proposed a new identification method for the continuous-time impulse response model (CTIRM) of a linear system from sampled data in the ill-posed cases of band-limited input, fast sampling rate and measurement noise of high level which may be colored.
1 Jan 1994
TL;DR: When performing the unconstrained optimisation of a complicated industrial problem, the main computational time is usually spent in the calculation of the objective function and its derivatives, so if a parallel processing machine is to be used, then a number of function evaluations must be calculated in parallel.
Abstract: When performing the unconstrained optimisation of a complicated industrial problem, the main computational time is usually spent in the calculation of the objective function and its derivatives. The calculation of the objective function is usually performed sequentially, so if a parallel processing machine is to be used, then either a number of function evaluations must be calculated in parallel or the sequential calculation of the objective function replaced by a parallel calculation.
9 Oct 1994
TL;DR: This work presents the first results obtained from two implementations of a hybrid architecture which balances exploration and exploitation to solve mazes with continuous search spaces, based around a Radial Basis Function (RBF) Neural Network.
Abstract: We present the first results obtained from two implementations of a hybrid architecture which balances exploration and exploitation to solve mazes with continuous search spaces. In both cases the critic is based around a Radial Basis Function (RBF) Neural Network which uses Temporal Difference learning to acquire a continuous valued internal model of the environment through interaction with it. Also in both cases an Evolutionary Algorithm is employed in the search policy for each movement. In the first implementation a Genetic Algorithm (GA) is used, and in the second an Evolutionary Strategy (ES). Over successive trials the maze solving agent learns the V-function, a mapping between real numbered positions in the maze and the value of being at those positions.
25 May 1994
TL;DR: In this article, the authors used a radial basis function (RBF) network to identify an oven with two inputs, one for heating and one for cooling, and used the internal model control as their control strategy.
Abstract: Many authors have shown by simulated studies, that a great number of non-linear dynamical systems could be identified and controlled by using neural network models. The authors applied these results to a real process : an oven with two inputs, one for heating and one for cooling. The output to be controlled is the temperature inside the oven. The choice of the control strategy on the one hand and the choice of the neural network architecture for the oven identification on the other hand had to satisfy two main objectives. First, the control strategy should be quite insensible to random disturbances (air leaks, door openings, ...) and the neural model should be able to fit any modification of the plant dynamics during its lifetime (modification of the internal load alteration of the heating system ...). The authors chose to use the internal model control as their control strategy. They also used a radial basis function network to identify the plant. The process identification is composed of two phases, an off-line one and an online one. The off-line part consists first in training the network while determining its internal structure using an initial training set. The on-line phase is the adaptive part of the control scheme.<
>
16 Jul 1994
TL;DR: Lower bounds on the VC-dimension proportional to the number of parameters for several function classes including two-layer neural networks with certain smooth activation functions and radial basis functions with a gaussian basis are obtained.
Abstract: We examine the relationship between the VC-dimension and the number of parameters of a smoothly parametrized function class. We show that the VC-dimension of such a function class is at least k if there exists a k-dimensional differentiable manifold in the parameter space such that each member of the manifold corresponds to a different decision boundary. Using this result, we are able to obtain lower bounds on the VC-dimension proportional to the number of parameters for several function classes including two-layer neural networks with certain smooth activation functions and radial basis functions with a gaussian basis. These lower bounds hold even if the magnitudes of the parameters are restricted to be arbitarily small. In Valiant's probably approximately correct learning framework, this implies that the number of example necessary for learning these function classes is at least linear in the number of parameters.
31 Oct 1994
TL;DR: The DNN and the modified RBF (MRBF) are applied to automatically classify 14 categories of land cover using multispectral aerial images and the performances are better than obtained by linear discriminant analysis (LDA) and backpropagation.
Abstract: Two methods of classification and related fast training algorithms are compared with each other and with backpropagation in this paper. The first method is the discriminant neural network (DNN). One hidden node is added at each design stage until the DNN meets the design requirements. The second method uses the radial basis function network (RBF). We modify the RBF by solving a succession of binary classification problems in order to provide fast training. These two classification methods are applied to automatically classify 14 categories of land cover using multispectral aerial images. We find that the training times for the DNN and the modified RBF (MRBF) are much less than the training times for backpropagation or RBF. The performances of DNN (72%) and MRBF (60%) are better than obtained by linear discriminant analysis (LDA) (55%). The resulting structure and computations are simpler for the DNN than for the other methods. >
29 Jun 1994
TL;DR: A new methodology to find the optimum width parameters in the RBF network model is presented, which combines genetic algorithms and the orthogonal least squares method.
Abstract: Radial basis function (RBF) networks are an attractive tool for modeling dynamic systems for control purposes. This paper presents a new methodology to find the optimum width parameters in the RBF network model. This methodology, which combines genetic algorithms and the orthogonal least squares method, is described in detail. Finally, two examples illustrate the usefulness of this method.
TL;DR: The NARX model is parametrized by the radial basis function neural network, and the parameters of the network are estimated with the orthogonal least-squares algorithm the MATLAB implementation of which is included.
6 Sep 1994
TL;DR: A new learning algorithm for radial basis functions (RBF) neural network, based on robust statistics, is presented, and the efficiency of the algorithm is shown in modelling two-dimensional functions.
Abstract: This paper presents a new learning algorithm for radial basis functions (RBF) neural network, based on robust statistics. The extention of the learning vector quantizer for second order statistics is one of the classical approaches in estimating the parameters of a RBF model. The paper provides a comparative study for these two algorithms regarding their application in probability density function estimation. The theoretical bias in estimating one-dimensional Gaussian functions are derived. The efficiency of the algorithm is shown in modelling two-dimensional functions. >