Abstract: We consider the problem of estimating the gradient lines of a density, which can be used to cluster points sampled from that density, for example via the mean-shift algorithm of Fukunaga and Hostetler (1975). We prove general convergence bounds that we then specialize to kernel density estimation.

Abstract: The optimum choice of weighting function for smoothing sample density func tions is discussed in the cases : (i) probability densities ; (ii) spectral densities. It is shown that the bias contribution to the mean-square error can in principle be eliminated to any required order, though the resulting theoretical gain in efficiency may not be realised except for very large samples. The relation with recent work by Rosenblatt, Daniels and Parzen is noted. The further problem of estimating the spectra of stationary point processes is also considered.

Abstract: Univariate Birnbaum-Saunders models have been widely applied to fatigue studies. Calculation of fatigue life is of great importance in determining the reliability of materials. We propose and derive new multivariate generalized Birnbaum-Saunders regression models. We use the maximum likelihood method and the EM algorithm to estimate their parameters. We carry out a simulation study to evaluate the performance of the corresponding maximum likelihood estimators. We illustrate the new models with real-world multivariate fatigue data.

Abstract: A novel approach is proposed for fast anomaly detection by one-class classification. Standard kernel density estimation is first used to obtain an estimate of the input probability density function, based on the one-class input data. This can be used for anomaly detection: query points are classed as anomalies if their density is below some threshold. The disadvantage is that kernel density estimation is lazy, that is the bulk of the computation is performed at query time. For large datasets it can be slow. Therefore it is proposed to approximate the density function using genetic programming symbolic regression, before imposing the threshold. The runtime of the resulting genetic programming trees does not depend on the size of the training data. The method is tested on datasets including in the domain of network security. Results show that the genetic programming approximation is generally very good, and hence classification accuracy approaches or equals that when using kernel density estimation to carry out one-class classification directly. Results are also generally superior to another standard approach, one-class support vector machines.

Abstract: Nowadays, one can find a huge set of methods to estimate the density function of a random variable nonparametrically Since the first version of the most elementary nonparametric density estimator (the histogram) researchers produced a vast amount of ideas especially corresponding to the issue of choosing the bandwidth parameter in a kernel density estimator model To focus not only on a descriptive application, the model seems to be quite suitable for application in discriminant analysis, where (multivariate) class densities are the basis for the assignment of a vector to a given class Thisarticle gives insight to most popular bandwidth parameter selectors as well as to the performance of the kernel density estimator as a classification method compared to the classical linear and quadratic discriminant analysis, respectively Both a direct estimation in a multivariate space as well as an application of the concept to marginal normalizations of the single variables will be taken into consideration From this report the gap between theory and application is going to be pointed out

Abstract: Standard kernel density estimation methods are very often used in practice to estimate density functions. It works well in numerous cases. However, it is known not to work so well with skewed, multimodal and heavy-tailed distributions. Such features are usual with income distributions, defined over the positive support. In this paper, we show that a preliminary logarithmic transformation of the data, combined with standard kernel density estimation methods, can provide a much better fit of the density estimation.

Abstract: The log-normal (LN) kernel estimator of a density with support [0, ∞) was discussed by Jin and Kawczak (2003). The contribution of this paper is to suggest a new class of LN kernel estimators using the idea of weighted distribution. The asymptotic properties of the new class of estimators are studied. Also, numerical studies based on both simulated and real data set are presented.

Abstract: This paper studies probability density estimation on the Siegel space. The Siegel space is a generalization of the hyperbolic space. Its Riemannian metric provides an interesting structure to the Toeplitz block Toeplitz matrices that appear in the covariance estimation of radar signals. The main techniques of probability density estimation on Riemannian manifolds are reviewed. For computational reasons, we chose to focus on the kernel density estimation. The main result of the paper is the expression of Pelletier’s kernel density estimator. The computation of the kernels is made possible by the symmetric structure of the Siegel space. The method is applied to density estimation of reflection coefficients from radar observations.

Abstract: Data-based choice of the bandwidth is an important problem in kernel density estimation. The pseudo-likelihood and the least-squares cross-validation bandwidth selectors are well known, but widely criticized in the literature. For heavy-tailed distributions, the L1 distance between the pseudo-likelihood-based estimator and the density does not seem to converge in probability to zero with increasing sample size. Even for normal-tailed densities, the rate of L1 convergence is disappointingly slow. In this article, we report an interesting finding that with minor modifications both the cross-validation methods can be implemented effectively, even for heavy-tailed densities. For both these estimators, the L1 distance (from the density) are shown to converge completely to zero irrespective of the tail of the density. The expected L1 distance also goes to zero. These results hold even in the presence of a strongly mixing-type dependence. Monte Carlo simulations and analysis of the Old Faithful geyser data sugge...

Abstract: Security issues have become increasingly important within distribution systems, which have led to the development of event detection algorithms (EDAs) to provide timely detection of intrusion events. The current study develops a localized model-based event detection algorithm that utilizes nonspecific water quality sensors to identify water quality anomalies. The proposed EDA focuses on evaluating a series of multivariate error signals between the observed signals and the model estimated signals based on a moving time-window of error statistics. The likelihood of the multivariate error signals is estimated using the product of univariate kernel density estimation (KDE), which is a type of nonparametric representation of the error distribution. A comprehensive analysis was performed using synthetic events to explore the combination of the moving window-pairs and bandwidth with respect to three injection strengths and two injection durations. In addition to the synthetic events, the EDA was also eva...

Abstract: In this paper, we analyze methods for estimating the density of a conditional expectation. We compare an estimator based on a straightforward application of kernel density estimation to a bias-corrected estimator that we propose. We prove convergence results for these estimators and show that the bias-corrected estimator has a superior rate of convergence. In a simulated test case, we show that the bias-corrected estimator performs better in a practical example with a realistic sample size.

Abstract: This paper proposed a nonparametric estimator for probability mass function of multivariate data. The estimator is based on discrete multivariate associated kernel without correlation structure. For the choice of the bandwidth diagonal matrix, we presented the Bayes global method against the likelihood cross-validation one, and we used the Bayesian Markov chain Monte Carlo (MCMC) method for deriving the global optimal bandwidth. We have compared the proposed method with the cross-validation method. The performance of both methods is evaluated under the integrated square error criterion through simulation studies based on for univariate and multivariate models. We also presented applications of the proposed methods to bivariate and trivariate real data. The obtained results show that the Bayes global method performs better than cross-validation one, even for the Poisson kernel which is the very bad discrete associated kernel among binomial, discrete triangular and Dirac discrete uniform kernels.

Abstract: A unified framework to analyse multivariate kernel estimators of distribution and survival functions is introduced, before turning our attention to receiver operating characteristic (ROC) curves. These are well-established visual analytic tools for univariate data samples, though their generalisation to multivariate data has been limited. Since non-parametric multivariate kernel smoothing methods possess excellent visualisation properties, they serve as a solid basis for their estimation. With optimal data-based bandwidth matrix selectors, we demonstrate that they possess suitable properties for exploratory data analysis of simulated and experimental data.

Abstract: The beta kernel estimator for a density with support was discussed by Chen [(1999) ‘Beta Kernel Estimators for Density Functions’, Computational Statistics and Data Analysis, 31, 131–145]. In this paper, when the underlying density has a fourth-order derivative, we improve the beta kernel estimator using the bias correction techniques based on two beta kernel estimators with different smoothing parameters. As a result, we propose new bias corrected beta kernel estimators involving the digamma functions, and then establish their asymptotic properties. Simulation studies are conducted to illustrate the finite sample performance of the proposed estimators.

Abstract: Estimating the derivatives of probability density functions is an essential step in statistical data analysis. A naive approach to estimate the derivatives is to first perform density estimation and then compute its derivatives. However, this approach can be unreliable because a good density estimator does not necessarily mean a good density derivative estimator. To cope with this problem, in this letter, we propose a novel method that directly estimates density derivatives without going through density estimation. The proposed method provides computationally efficient estimation for the derivatives of any order on multidimensional data with a hyperparameter tuning method and achieves the optimal parametric convergence rate. We further discuss an extension of the proposed method by applying regularized multitask learning and a general framework for density derivative estimation based on Bregman divergences. Applications of the proposed method to nonparametric Kullback-Leibler divergence approximation and bandwidth matrix selection in kernel density estimation are also explored.

Abstract: The estimation of probability densities based on available data is a central task in many statistical applications. Especially in the case of large ensembles with many samples or high-dimensional sample spaces, computationally efficient methods are needed. We propose a new method that is based on a decomposition of the unknown distribution in terms of so-called distribution elements (DEs). These elements enable an adaptive and hierarchical discretization of the sample space with small or large elements in regions with smoothly or highly variable densities, respectively. The novel refinement strategy that we propose is based on statistical goodness-of-fit and pair-wise (as an approximation to mutual) independence tests that evaluate the local approximation of the distribution in terms of DEs. The capabilities of our new method are inspected based on several examples of different dimensionality and successfully compared with other state-of-the-art density estimators.

Abstract: We consider the nonparametric estimation of the univariate heavy tailed probability density function (pdf) with a support on $[0,\infty)$ by independent data. To this end we construct the new kernel estimator as a combination of the asymmetric gamma and weibull kernels, ss. gamma-weibull kernel. The gamma kernel is nonnegative, changes the shape depending on the position on the semi-axis and possess good boundary properties for a wide class of densities. Thus, we use it to estimate the pdf near the zero boundary. The weibull kernel is based on the weibull distribution which can be heavy tailed and hence we use it to estimate the tail of the unknown pdf. The theoretical asymptotic properties of the proposed density estimator like bias and variance are derived. We obtain the optimal bandwidth selection for the estimate as a minimum of the mean integrated squared error (MISE). Optimal rate of convergence of the MISE for the density is found.

Abstract: Kernel type density estimators are studied for random fields A functional central limit theorem in the space of square integrable functions is proved if the locations of observations become more and more dense in an increasing sequence of domains

Abstract: Scatter plots of multivariate data sets motivate modeling of star-shaped distributions beyond elliptically contoured ones. We study properties of estimators for the density generator function, the star-generalized radius distribution and the density in a star-shaped distribution model. For the generator function and the star-generalized radius density, we consider a non-parametric kernel-type estimator. This estimator is combined with a parametric estimator for the contours which are assumed to follow a parametric model. Therefore, the semiparametric procedure features the flexibility of nonparametric estimators and the simple estimation and interpretation of parametric estimators. Alternatively, we consider pure parametric estimators for the density. For the semiparametric density estimator, we prove rates of uniform, almost sure convergence which coincide with the corresponding rates of one-dimensional kernel density estimators when excluding the center of the distribution. We show that the standardized density estimator is asymptotically normally distributed. Moreover, the almost sure convergence rate of the estimated distribution function of the star-generalized radius is derived. A particular new two-dimensional distribution class is adapted here to agricultural and financial data sets.

Abstract: This paper studies how to sample load more realistically and efficiently for security constraint unit commitment (SCUC) problems in order to achieve a high degree of robustness of the unit commitment (UC) solution. For example, given the UC solution, 95% of load profiles can be supplied. Principal component analysis (PCA) is introduced to find a clear feature of the historical load in two-dimensional space rather than the original high-dimensional load space, whose feature is hard to capture. Kernel density estimation (KDE) with Gaussian kernel is applied to form the probability density function (PDF), from which hourly load profiles are sampled. The load profiles based on sparse sampling are used to find the UC solution with high robustness while the load profiles based on dense sampling are used to verify the robustness of the obtained UC solution.