Abstract: Boundary effects are well known to occur in nonparametric density estimation when the support of the density has a finite endpoint. The usual kernel density estimators require modifications when estimating the density near endpoints of the support. In this paper, we propose a new and intuitive method of removing boundary effects in density estimation. Our idea, which replaces the unwanted terms in the bias expansion by their estimators, offers new ways of constructing boundary kernels and optimal endpoint kernels. We also discuss the choice of bandwidth variation functions at the boundary region. The performance of our results are numerically analyzed in a Monte Carlo study.

Abstract: The IDEA framework is a general framework for iterated density estimation evolutionary algorithms. These algorithms use probabilistic models to guide the search in stochastic optimization. The estimation of densities for subsets of selected samples and the sampling from the resulting distributions, is the combination of the evolutionary recombination and mutation steps used in EAs. We investigate how the normal kernels probability density function can be used as the distribution of the problem variables in order to perform optimization using the IDEA framework. As a result, we present three probability density structure search algorithms, as well as a general estimation and samphng algorithm, all of which use the normal kernels distribution

Abstract: The application of nonparametric probability density function estimation for the purpose of data analysis is well established. More recently, such methods have been applied to fluid flow calculations since the density of the fluid plays a crucial role in determining the flow. Furthermore, when the calculations involve directional or axial data, the domain of interest falls on the surface of the sphere. Accurate and fast estimation of probability density functions is crucial for these calculations since the density estimation is performed at each iteration during the computation. In particular the values fn (X1 ), fn (X2), ... , fn (Xn) of the density estimate at the sampled points Xi are needed to evolve the system. Usual nonparametric estimators make use of kernel functions to construct fn. We propose a special sequence of weight functions for nonparametric density estimation that is especially suitable for such applications. The resulting method has a computational advantage over kernel methods in certain situations and also parallelizes easily. Conditions for convergence turn out to be similar to those required for kernel-based methods. We also discuss experiments on different distributions and compare the computational efficiency of our method with kernel based estimators.

Abstract: There is a tendency for the true variability of feasible GLS estimators to be understated by asymptotic standard errors. For estimation of SUR models, this tendency becomes more severe in large equation systems when estimation of the error covariance matrix, C, becomes problematic. We explore a number of potential solutions involving the use of improved estimators for the disturbance covariance matrix and bootstrapping. In particular, Ullah and Racine (1992) have recently introduced a new class of estimators for SUR models that use nonparametric kernel density estimation techniques. The proposed estimators have the same structure as the feasible GLS estimator of Zellner (1962) differing only in the choice of estimator for C. Ullah and Racine (1992) prove that their nonparametric density estimator of C can be expressed as Zellner's original estimator plus a positive definite matrix that depends on the smoothing parameter chosen for the density estimation. It is this structure of the estimator that most int...

Abstract: We present a new algorithm for the estimation of probability density functions (PDF). This founds a large number of applications in the context of statistical signal processing problems, such as detection, estimation, filtering or pattern recognition and classification. Our approach relies on the QQ-plot technique. The estimates of the first and second order statistics of the observed random data are used together with a suboptimal piecewise linear approximation of the QQ-plot, yielding a new class of PDF estimator. We describe the algorithm and test it in comparison with other techniques, showing that our approach provides better results.

Abstract: As is well-known, a heteroskedasticity and autocorrelation consistent covariance matrix is proportional to a spectral density matrix at frequency zero and can be consistently estimated by such popular kernel methods as those of Andrews-Newey-West. In practice, it is difficult to estimate the spectral density matrix if it has a peak at frequency zero, which can arise when there is strong autocorrelation, as often encountered in economic and financial time series. Kernels, as a local averaging method, tend to underestimate the peak, thus leading to strong overrejection in testing and overly narrow confidence intervals in estimation. As a new mathematical tool generalizing Fourier transform, wavelet transform is a powerful tool to investigate such local properties as peaks and spikes, and thus is suitable for estimating covariance matrices. In this paper, we propose a class of wavelet estimators for the covariance matrices of econometric parameter estimators. We show the consistency of the wavelet-based covariance estimators and derive their asymptotic mean squared errors, which provide insight into the smoothing nature of wavelet estimation. We propose a data-driven method to select the finest scale---the smoothing parameter in wavelet estimation, making the wavelet estimators operational in practice. A simulation study compares the finite sample performances of the wavelet estimators and the kernel counterparts. As expected, the wavelet method outperforms the kernel method when there exists relatively strong autocorrelation in the data.

Abstract: We introduce a new class of nonparametric density estimators. It includes the classical kernel density estimator as well as the popular Abramson's estimator. We show that generalized estimators may perform much better than the classical one if the distribution has a heavy tail. The asymptotics of the mean squared error (MSE), optimal (in a sense) kernel, and smoothing parameter are found.

Abstract: Part 1 Trend estimation in correlated noise, Rudolf Beran: the lumber-thickness data asymptotically minimax estimators proofs Part 2 Higher order analysis at Lebesgue points, A Berlinet and S Levallois: definitions and main results two examples with infinite derivatives example with discontinuity of the second kind conclusion Part 3 Regression analysis for multivariate failure time observations, T Cai and LJ Wei: estimation of regression parameters simultaneous predictions of survival probabilities and related quantities example appendix -asymptotic joint distribution of {b*k} with discrete covariates Part 4 Local estimation of a biometric function with covariate effects, Zongwu Cai and Lianfen Qian: local likelihood method an exponential regression approach simulation study appendix Part 5 The estimation of conditional densities, X Chen et al: kernel conditional density estimates asymptotic theory of conditional density estimates and bandwidth choice Part 6 Functional limit theorems for induced order statistics of a sample from a domain of attraction of a-stable law, a I (0,2), Yu Davydov and V Egorov: notation auxillary facts main results lemmas proofs concluding remarks Part 7 Limit laws for kernel density estimators for kernels with unbounded supports, Paul Deheuvel: introduction and results proofs Part 8 Inequalities for a new data-based method for selecting nonparametric density estimates, Luc Devroye et al: the basic estimate standard kernel estimate -Riemann kernels standard kernel estimate - general kernels multiparameter kernel estimates - product kernels multiparameter kernel estimates - ellipsoidal kernels the transformed kernel estimate Monte Carlo simulations proof of theorem 1 Part 9 B-fuzzy stochastics, CA Drossos et al: preliminaries basic results B-fuzzy structures B-fuzzy statistics Part 10 Detecting jumps in nonparametric regression, Ch Dubowik and U Stadtmuller: the proposed model new results simulations appendix (Part contents)

Abstract: This paper studies boundary effects of the kernel density estimation and proposes some remedies to the problems. Since the kernel estimate is designed for estimating a smooth density, it introduces a large bias near the boundaries where the density is discontinuous. Bandwidth selectors developed for the kernel estimate that select a small bandwidth to reduce the bias can dramatically increase the variation and roughness of the density estimate. In this paper, several boundary adjusted procedures for estimating the density, as well as selecting the bandwidth, are introduced. The proposed procedures greatly reduce the boundary effects and is shown that these density estimates have the same optimal convergence rate as that of the kernel density estimate of a smooth density. Some asymptotic results about the boundary adjusted procedures are provided. Simulation studies were carried out to check the empiric performance of the proposed procedures compared to some existing boundary-corrected estimation procedures. In general, simulation results indicate that for moderate to large sample sizes, the proposed procedures reduce the boundary effects substantially, and are better than comparable existing methods. As an example, we estimate a relevant density connected with some coal-mining disaster data.

Abstract: This thesis consists of three papers (Papers A-C) on problems in nonparametric functional estimation, in particular density and regression function estimation and deconvolution, under order assumptions. Pointwise limit distribution results are stated for the obtained estimators, which include isotonic regression estimates, nonparametric maximum likelihood estimates of monotone densities, estimates of convex regression and density functions and deconvolution estimates. Paper A states a limit distribution formula for the greatest convex minorant mapping and its derivative, which is then applied to regression function and density function estimation under monotonicity or convexity restrictions, at points of continuity and under various smoothness assumptions on the unknown function. Also treated is isotonization of kernel estimates, with application to regression and density estimation. Paper B extends the results of Paper A to limit results at points of discontinuity of the unknown function. Paper C is concerned with deconvolution under order restrictions. Our methods give a unified approach to regression and density function estimation with order restrictions, thereby restating many previously known results as special cases as well as obtaining new results, mainly for dependent data and/or discontinuous functions. (Less)

Abstract: We consider deconvolution problems where the observations Y are equal in distribution to X+Z with X and Z independent random variables. The distribution of Z is assumed to be known and X has an unknown probability density that we want to estimate. The case where Z has a known Laplace distribution is investigated in detail. We consider an estimator that is the sum of two kernel estimators and investigate the gain to be achieved when we use different bandwidths instead of equal bandwidths. In less detail we review exponential deconvolution and estimation of a linear combination of density derivatives. We derive expansions for the asymptotic mean integrated squared error, asymptotically optimal bandwidths as well as a formula for the ratio of the smallest asymptotic error of the multiple bandwidth and equal bandwidth estimator. The finite sample performance of the multi bandwidth kernel estimators is investigated by computation of the exact mean integrated squared error for several target densities.

Abstract: In this paper we consider two important topics: density estimation and random variate generation. We will present a framework that is easily implemented using the familiar multilayer neural network. First, we develop two new methods for density estimation, a stochastic method and a related deterministic method. Both methods are based on approximating the distribution function, the density being obtained by differentiation. In the second part of the paper, we develop new random number generation methods. Our methods do not suffer from some of the restrictions of existing methods in that they can be used to generate numbers from an observed inverse relationship between the density estimation process and the random number generation process. We present two variants of this method -- a stochastic and a deterministic version. We propose a second method that is based on formulating the task as a control problem, where a "controller network" is trained to shape a given density into the desired density. We justify the use of all the methods that we propose by providing theoretical convergence results. In particular, we prove that the L8 convergence to the true density to both the density estimation and random variate generation techniques occurs as a rate O((log log N/N)^((1-e)/2) where N is the number of data points and e can be made arbitrarily small for sufficiently smooth target densities. This bound is very close to the optimally achievable convergence rate under similar smoothness conditions. Also, for comparison, the L2 (RMS) convergence rate of a positive kernel density estimator is O(N^(-2/5)) when the optimal kernel width is used. We present numerical simulations to illustrate the performance of the proposed density estimation and random variate generation methods. In addition, we present an extended introduction and bibliography that serves as an overview and reference for the practitioner.

Abstract: A new approach to density-based clustering and unsupervised classification is introduced for topographic maps. By virtue of our topographic map learning rule, called the kernel-based Maximum Entropy learning Rule (kMER), all neurons have an equal probability to be active (equiprobabilistic map) and, in addition, pilot density estimates are obtained that are compatible with the variable kernel density estimation method. The neurons receive their cluster labels by performing hill-climbing on the density estimates which are located at the neuron weights only. Several methods are suggested and explored for determining the cluster boundaries and the clustering performance is tested for the case where the cluster regions are used for unsupervised classification purposes. Finally, the difference is indicated between (Gaussian) kernel-based density modeling with kMER, and (Gaussian) mixture modeling with maximum likelihood learning.

Abstract: We find a general formula for moments of spherically symmetric multivariate kernels, and provide details for kernels in common use.

Abstract: To extract multivariate probability density functions (PDF) from a clustered training data set for condition monitoring purposes, a modified kernel density estimation method is suggested using regularisation and deconvolution techniques. Case studies show that it is a useful pragmatic method for real industrial data.

Abstract: A new method for computing the probability distribution of a given sample of data is proposed. The observations are mapped into the finite interval [-1,1] and a shape-preserving spline is used to calculate the derivative of the cumulative distribution function. Although based on a spline, the procedure guarantees non-negative density estimates. The method is compared to a normal kernel with plug-in bandwidth for a range of test distributions. As well as requiring less computational effort, the performance of the spline estimate of density is marginally superior to that of the kernel for distributions that have an infinite domain, but is currently inferior to second generation kernels for semi-infinite domains.

Abstract: This paper deals with nonparametric estimation of the boundary curve of the support of a bivariate density function. This estimation problem arises in various contexts, such as for example scatterpoint image analysis and frontier estimation in econometrics. The setup in this paper is a general one, allowing the bivariate density function to be infinite, bounded away from zero or zero at the boundary. Two estimators for the boundary curve are introduced, both based on order statistics. The asymptotic distribution of the estimators and their rate of convergence are established. Via a comparison of the rates of convergence we recommend which estimator to use in a particular situation. Both estimators can be used as an initial estimator in a two-stage procedure, designed for getting a better estimation. Simulation studies demonstrate the finite-sample behavior of the estimators and the proposed two-stage procedure. We illustrate the procedure on a data set on American electric utility companies.

Abstract: A new procedure is proposed for deriving variable bandwidths in univariate kernel density estimation, based upon likelihood cross-validation and an analysis of a Bayesian graphical model. The procedure admits bandwidth selection which is flexible in terms of the amount of smoothing required. In addition, the basic model can be extended to incorporate local smoothing of the density estimate. The method is shown to perform well in both theoretical and practical situations, and we compare our method with those of Abramson (The Annals of Statistics 10: 1217–1223) and Sain and Scott (Journal of the American Statistical Association 91: 1525–1534). In particular, we note that in certain cases, the Sain and Scott method performs poorly even with relatively large sample sizes. We compare various bandwidth selection methods using standard mean integrated square error criteria to assess the quality of the density estimates. We study situations where the underlying density is assumed both known and unknown, and note that in practice, our method performs well when sample sizes are small. In addition, we also apply the methods to real data, and again we believe our methods perform at least as well as existing methods.

Abstract: Transformation from a parametrized family can be combined with kernel density estimation for improved effectiveness. Pilot estimators had been proposed for the parameter that gives the optimal transformation, yet their rates of convergence had not been resolved. In this paper, the rates of convergence are given. An improved estimator is also proposed which achieves the desirable root-n rate of convergence.