Abstract: We address the problem of density estimation with $\mathbb{L}_s$-loss by selection of kernel estimators. We develop a selection procedure and derive corresponding $\mathbb{L}_s$-risk oracle inequalities. It is shown that the proposed selection rule leads to the estimator being minimax adaptive over a scale of the anisotropic Nikol'skii classes. The main technical tools used in our derivations are uniform bounds on the $\mathbb{L}_s$-norms of empirical processes developed recently by Goldenshluger and Lepski [Ann. Probab. (2011), to appear].

Abstract: The notion of probability density for a random function is not as straightforward as in finite-dimensional cases. While a probability density function generally does not exist for functional data, we show that it is possible to develop the notion of density when functional data are considered in the space determined by the eigenfunctions of principal component analysis. This leads to a transparent and meaningful surrogate for density defined in terms of the average value of the logarithms of the densities of the distributions of principal components for a given dimension. This density approximation is estimable readily from data. It accurately represents, in a monotone way, key features of small-ball approximations to density. Our results on estimators of the densities of principal component scores are also of independent interest; they reveal interesting shape differences that have not previously been considered. The statistical implications of these results and properties are identified and discussed, and practical ramifications are illustrated in numerical work.

Abstract: We present a real data set of claims amounts where costs related to damage are recorded separately from those related to medical expenses. Only claims with positive costs are considered here. Two approaches to density estimation are presented: a classical parametric and a semi-parametric method, based on transformation kernel density estimation. We explore the data set with standard univariate methods. We also propose ways to select the bandwidth and transformation parameters in the univariate case based on Bayesian methods. We indicate how to compare the results of alternative methods both looking at the shape of the overall density domain and exploring the density estimates in the right tail.

Abstract: Orthogonal series density estimation is a powerful nonparametric estimation methodology that allows one to analyze and present data at hand without any prior opinion about shape of an underlying density. The idea of construction of an adaptive orthogonal series density estimator is explained on the classical example of a direct sample from a univariate density. Data-driven estimators, which have been used for years, as well as recently proposed procedures, are reviewed. Orthogonal series estimation is also known for its sharp minimax properties which are explained. Furthermore, applications of the orthogonal series methodology to more complicated settings, including censored and biased data as well as estimation of the density of regression errors and the conditional density, are also presented. Copyright © 2010 John Wiley & Sons, Inc.

Abstract: An approach for multivariate statistical monitoring based on kernel independent component analysis (Kernel ICA) is presented. Different from the recently developed KICA which means kernel principal component analysis (KPCA) plus independent component analysis (ICA), Kernel ICA is an improvement of ICA and uses contrast functions based on canonical correlations in a reproducing kernel Hilbert space. The basic idea is to use Kernel ICA to extract independent components and later to provide enhanced monitoring of multivariate processes. I2 (the sum of the squared independent scores) and squared prediction error (SPE) are adopted as statistical quantities. Besides, kernel density estimation (KDE) is described to calculate the confidence limits. The proposed monitoring method is applied to fault detection in the simulation benchmark of the wastewater treatment process and the Tennessee Eastman process, the simulation results clearly show the advantages of Kernel ICA monitoring in comparison to ICA and KICA monitoring.

Abstract: In this paper, we present a new adaptive-scale kernel consensus (ASKC) robust estimator as a generalization of the popular and state-of-the-art robust estimators such as random sample consensus (RANSAC), adaptive scale sample consensus (ASSC), and maximum kernel density estimator (MKDE). The ASKC framework is grounded on and unifies these robust estimators using nonparametric kernel density estimation theory. In particular, we show that each of these methods is a special case of ASKC using a specific kernel. Like these methods, ASKC can tolerate more than 50 percent outliers, but it can also automatically estimate the scale of inliers. We apply ASKC to two important areas in computer vision, robust motion estimation and pose estimation, and show comparative results on both synthetic and real data.

Abstract: Nonparametric kernel estimators are widely used in many research ar- eas of statistics. An important nonparametric kernel estimator of a regression function is the Nadaraya-Watson kernel regression estima- tor which is often obtained by using a fixed bandwidth. However, the adaptive kernel estimators with varying bandwidths are specially used to estimate density of the long-tailed and multi-mod distributions. In this paper, we consider the adaptive Nadaraya-Watson kernel regression estimators. The results of the simulation study show that the adaptive Nadaraya-Watson kernel estimators have better performance than the kernel estimations with fixed bandwidth.

Abstract: This paper introduces a class of k-nearest neighbor ($k$-NN) estimators called bipartite plug-in (BPI) estimators for estimating integrals of non-linear functions of a probability density, such as Shannon entropy and R\'enyi entropy. The density is assumed to be smooth, have bounded support, and be uniformly bounded from below on this set. Unlike previous $k$-NN estimators of non-linear density functionals, the proposed estimator uses data-splitting and boundary correction to achieve lower mean square error. Specifically, we assume that $T$ i.i.d. samples ${X}_i \in \mathbb{R}^d$ from the density are split into two pieces of cardinality $M$ and $N$ respectively, with $M$ samples used for computing a k-nearest-neighbor density estimate and the remaining $N$ samples used for empirical estimation of the integral of the density functional. By studying the statistical properties of k-NN balls, explicit rates for the bias and variance of the BPI estimator are derived in terms of the sample size, the dimension of the samples and the underlying probability distribution. Based on these results, it is possible to specify optimal choice of tuning parameters $M/T$, $k$ for maximizing the rate of decrease of the mean square error (MSE). The resultant optimized BPI estimator converges faster and achieves lower mean squared error than previous $k$-NN entropy estimators. In addition, a central limit theorem is established for the BPI estimator that allows us to specify tight asymptotic confidence intervals.

Abstract: We derive asymptotic normality of kernel type deconvolution density estimators. In particular, we consider deconvolution problems where the known component of the convolution has a symmetric λ-stable distribution with 0 < λ ≤ 2. It turns out that the limit behavior changes if the exponent parameter λ passes the value 1, the case of Cauchy deconvolution.

Abstract: The beta kernel estimators are shown in Chen [S.X. Chen, Beta kernel estimators for density functions, Comput. Statist. Data Anal. 31 (1999), pp. 131–145] to be non-negative and have less severe boundary problems than the conventional kernel estimator. Numerical results in Chen [S.X. Chen, Beta kernel estimators for density functions, Comput. Statist. Data Anal. 31 (1999), pp. 131–145] further show that beta kernel estimators have better finite sample performance than some of the widely used boundary corrected estimators. However, our study finds that the numerical comparisons of Chen are confounded with the choice of the bandwidths and the quantities being compared. In this paper, we show that the performances of the beta kernel estimators are very similar to that of the reflection estimator, which does not have the boundary problem only for densities exhibiting a shoulder at the endpoints of the support. For densities not exhibiting a shoulder, we show that the beta kernel estimators have a serious boun...

Abstract: This paper presents a new nonparametric method for computing the conditional Value-at-Risk, based on a local approximation of the conditional density function in a neighborhood of a predetermined extreme value for univariate and multivariate series of portfolio returns. For illustration, the method is applied to intraday VaR estimation on portfolios of two stocks traded on the Toronto Stock Exchange. The performance of the new VaR computation method is compared to the historical simulation, variance-covariance, and J. P. Morgan methods.

Abstract: In this paper we propose a new nonparametric kernel-based estimator for a density function f which achieves bias reduction relative to the classical Rosenblatt–Parzen estimator. Contrary to some existing estimators that provide for bias reduction, our estimator has a full asymptotic characterisation including uniform consistency and asymptotic normality. In addition, we show that bias reduction can be achieved without the disadvantage of potential negativity of the estimated density – a deficiency that results from using higher order kernels. Our results are based on imposing global Lipschitz conditions on f and defining a novel corresponding kernel. A Monte Carlo study is provided to illustrate the estimator's finite sample performance.

Abstract: A double transformation kernel density estimator that is suitable for heavy-tailed distributions is presented. Using a double transformation, an asymptotically optimal bandwidth parameter can be calculated when minimizing the expression of the asymptotic mean integrated squared error of the transformed variable. Simulation results are presented showing that this approach performs better than existing alternatives. An application to insurance claim cost data is included.

Abstract: In the nonparametric kernel estimation of the unknown probability densities and their derivatives there exist several methods for estimation of the kernel function bandwidth of which the CV and SCV methods of cross-validation are most simple and suitable. The former method was developed both for the density itself and its derivatives; the latter one, for density only. Yet it generates estimates with a higher rate of convergence and substantially smaller scatter. For the problem of nonparametric restoration of the density derivative from an independent sample, a data-based estimate of the kernel function bandwidth was constructed.

Abstract: In this paper we consider the problem of parameter estimation for intracellular network models with statistical Bayesian approaches. We use systems of nonlinear differential equations in order to describe the dynamics of those networks. In this setting, the posterior distribution has to be investigated via Markov chain Monte Carlo sampling. An estimation of summary statistics of the posterior from these samples requires appropriate density estimation methods. We focus in this study particularly on the influence of kernel density estimators on the expected information content of the posterior. A new method for the calculation of this information content is introduced that uses directly the unnormalized posterior values at the sample points. We exemplarily show its superiority to kernel estimators on a model of secretory pathway control at the trans-Golgi network in mammalian cells.

Abstract: In decision and estimation fusion of multisensor systems, the joint density of random signal measurements of multiple sensors can be used to improve the performance of decision and estimation fusion. In sensor sampling, enough measurement samples can be obtained easily in each single sensor. However,it is very difficult to get enough synchronous joint measurement samples among sensors. The classic kernel estimation can only make use of the little joint measurement samples without using the enough marginal samples. In this paper,the authors propose two multivariate density estimation methods based on Copulas which can make good use of the joint measurement samples and enough marginal measurement samples simultaneously. Several simulation examples are presented. It shows that two methods are better than the classic kernel density estimation.

Abstract: We consider the problem of estimation of density function by the method of delta sequences for functional data with values in an infinite dimensional separable Banach space.

Abstract: A wavelet based linear estimator is proposed for the derivatives of a probability density function based on a sample from a finite mixture of components with varying mixing proportions. It extends the linear estimator of a probability density function proposed by Pokhyl’ko (Theor. Probability and Math. Statist, 70 (2005) 135–145). Upper bounds on L2 and L∞ losses are obtained for such estimators.

Abstract: A generalized or tunable-kernel model is proposed for probability density function estimation based on an orthogonal forward regression procedure. Each stage of the density estimation process determines a tunable kernel, namely, its center vector and diagonal covariance matrix, by minimizing a leave-one-out test criterion. The kernel mixing weights of the constructed sparse density estimate are finally updated using the multiplicative nonnegative quadratic programming algorithm to ensure the nonnegative and unity constraints, and this weight-updating process additionally has the desired ability to further reduce the model size. The proposed tunable-kernel model has advantages, in terms of model generalization capability and model sparsity, over the standard fixed-kernel model that restricts kernel centers to the training data points and employs a single common kernel variance for every kernel. On the other hand, it does not optimize all the model parameters together and thus avoids the problems of high-dimensional ill-conditioned nonlinear optimization associated with the conventional finite mixture model. Several examples are included to demonstrate the ability of the proposed novel tunable-kernel model to effectively construct a very compact density estimate accurately.

Abstract: We propose a nonparametric multiplicative bias corrected transformation estimator designed for heavy tailed data. The multiplicative correction is based on prior knowledge and has a dimension reducing effect at the same time as the original dimension of the estimation problem is retained. Adding a tail-flattening transformation improves the estimation significantly – particularly in the tail – and provides significant graphical advantages by allowing the density estimation to be visualized in a simple way. The combined method is demonstrated on a fire insurance data set and where it provides excellent performance in a data-driven simulation study.

Abstract: In this paper, a distance-based method for both multivariate non-parametric density and conditional density estimation is proposed. The contributions are the formulation of both density estimation problems as weight optimization problems for Gaussian mixtures centered about samples with identical parameters. Furthermore, the minimization is based on the modified Cramer-von Mises distance of the Localized Cumulative Distributions, removing the ambiguity of the definition of the multivariate cumulative distribution function. The minimization problem is amended with a regularization term penalizing the densities' roughness to avoid overfitting. The resulting estimation problems for both densities and conditional densities are shown to be phrasable in the form of readily implementable quadratic programs. Experimental comparison against EM, SVR, and GPR based on the log-likelihood and performance in benchmark recursive filtering applications show high quality of the densities and good performance at less computational cost, i.e., the density representations are sparser.

Abstract: In this work, we study the asymptotic properties of smoothed nonparametric kernel spectral density estimators for the spatial spectral density. We consider the case of continuous stationary spatial processes under a shrinking asymptotic framework. Expressions for the bias and the covariance structure are obtained and the implications for the edge effect bias of the choice of the kernel, bandwidth and spacing parameter in the design are also discussed, both for tapered and untapered estimates. Results are illustrated with a simulation study.

Abstract: This paper focuses on the bandwidth selection in the kernel density estimation in the univariate case. We compared via simulation between the optimal bandwidth selection based on the mean integrated squared error (MISE) and based on the asymptotic mean integrated squared error (AMISE) for various densities and sample sizes. Also, we compared between the MISE and the AMISE in the kernel density estimation. Through simulation, these optimal bandwidths are compared with the bandwidth selection using the methods least squared cross-validation (LSCV), biased cross-validation (BCV), and direct plug-in (DPI).

Abstract: In this paper an efficient approach to nonlinear non-Gaussian state estimation based on spline filtering is presented. The estimation of the conditional probability density of the unknown state can be ideally achieved through Bayes rule. However, the associated computational requirements make it impossible to implement this online filter in practice. In the general particle filtering problem, estimation accuracy increases with the number of particles at the expense of increased computational load. In this paper, B-Spline interpolation is used to represent the density of the state pdf through a low order continuous polynomial. The motivation is to reduce the computational cost. The motion of spline control points and corresponding coefficients is achieved through implementation of the Fokker-Planck equation, which describes the propagation of state probability density function between measurement instants. This filter is applicable for a general state estimation problem as no assumptions are made about the underlying probability density.