Showing papers on "Multivariate kernel density estimation published in 2003"
Proceedings Article•
1 Jan 2003
TL;DR: This paper presents an algorithm for kernel density estimation, the chief nonparametric approach, which is dramatically faster than previous algorithmic approaches in terms of both dataset size and dimensionality and is an instance of a new principle of algorithm design: multi-recursion, or higher-order algorithm design.
Abstract: Density estimation is a core operation of virtually all probabilistic learning methods (as opposed to discriminative methods). Approaches to density estimation can be divided into two principal classes, parametric methods, such as Bayesian networks, and nonparametric methods such as kernel density estimation and smoothing splines. While neither choice should be universally preferred for all situations, a well-known benefit of nonparametric methods is their ability to achieve estimation optimality for ANY input distribution as more data are observed, a property that no model with a parametric assumption can have, and one of great importance in exploratory data analysis and mining where the underlying distribution is decidedly unknown. To date, however, despite a wealth of advanced underlying statistical theory, the use of nonparametric methods has been limited by their computational intractibility for all but the smallest datasets. In this paper, we present an algorithm for kernel density estimation, the chief nonparametric approach, which is dramatically faster than previous algorithmic approaches in terms of both dataset size and dimensionality. Furthermore, the algorithm provides arbitrarily tight accuracy guarantees, provides anytime convergence, works for all common kernel choices, and requires no parameter tuning. The algorithm is an instance of a new principle of algorithm design: multi-recursion, or higher-order
215 citations
TL;DR: Akdensity as discussed by the authors extends the official kdensity that estimates density functions by the kernel method by allowing the use of an adaptive kernel approach with varying, rather than xed, bandwidths.
Abstract: T his insert describes the module akdensity. akdensity extends the official kdensity that estimates density functions by the kernel method. The extensions are of two types: akdensity allows the use of an adaptive kernel approach with varying, rather than xed, bandwidths; and akdensity estimates pointwise variability bands around the estimated density functions.
183 citations
Posted Content•
TL;DR: In this paper, the authors extend the class of non-negative, asymmetric kernel density estimators and propose Birnbaum-Saunders (BS) and lognormal (LN) kernel density functions.
Abstract: In this article we extend the class of non-negative, asymmetric kernel density estimators and propose Birnbaum-Saunders (BS) and lognormal (LN) kernel density functions. The density functions have bounded support on [0,1). Both BS and LN kernel estimators are free of boundary bias, non-negative, with natural varying shape, and achieve the optimal rate of convergence for the mean integrated squared error. We apply BS and LN kernel density estimators to high frequency intraday time duration data. The comparisons are made on several nonparametric kernel density estimators. BS and LN kernels perform better near the boundary in terms of bias reduction.
85 citations
TL;DR: The use of one implementation of a variable kernel estimator in conjunction with several rules and procedures for bandwidth selection applied to several real datasets showed fewer modes than those chosen by the Silverman test, especially those distributions in which multimodality was caused by several noisy minor modes.
Abstract: V ariable bandwidth kernel density estimators increase the window width at low densities and decrease it where data concentrate This represents an improvement over the xed bandwidth kernel density estimators In this ar- ticle, we explore the use of one implementation of a variable kernel estimator in conjunction with several rules and procedures for bandwidth selection applied to several real datasets The considered examples permit us to state that when work- ing with tens or a few hundreds of data observations, least-squares cross-validation bandwidth rarely produces useful estimates; with thousands of observations, this problem can be surpassed Optimal bandwidth and biased cross-validation (BCV), in general, oversmooth multimodal densities The Sheather
Jones plug-in rule pro- duced bandwidths that behave slightly better in this respect The Silverman test is considered as a very sophisticated and safe procedure to estimate the number of modes in univariate distributions; however, similar results could be obtained with the Sheather
Jones rule, but at a much lower computational cost As expected, the variable bandwidth kernel density estimates showed fewer modes than those chosen by the Silverman test, especially those distributions in which multimodality was caused by several noisy minor modes More research on the subject is needed
65 citations
TL;DR: In this article, a two-parameter Mobius-like function is proposed to map sample data drawn from a semi-infinite space into (-1,1) and a standard kernel method is then used to estimate the density.
Abstract: It is well known that the manipulation of sample data by means of a parametric function can improve the performance of kernel density estimation. This article proposes a two-parameter Mobius-like function to map sample data drawn from a semi-infinite space into (-1,1). A standard kernel method is then used to estimate the density. The proposed method is shown to yield effective estimates of density and is computationally more efficient than other well-known transformation methods. The efficacy of the technique is demonstrated in a practical setting by application to two datasets.
55 citations
TL;DR: In this article, an estimate of the mode of a multivariate probability density using a kernel estimate drawn from a random sample is proposed, defined by maximizing the kernel estimate over the set of sample values.
Abstract: The authors consider an estimate of the mode of a multivariate probability density using a kernel estimate drawn from a random sample. The estimate is defined by maximizing the kernel estimate over the set of sample values. The authors show that this estimate is strongly consistent and give an almost sure rate of convergence. This rate depends on the sharpness of the density near the true mode, which is measured by a peak index.
53 citations
Posted Content•
TL;DR: In this article, the authors give optimal sampling schemes of continuous time processes and study the effects of known or small errors in variables on such samplings, and various simulations are also presented.
Abstract: In the framework of nonparametric density estimation, first, we give optimal sampling schemes of continuous time processes. Next, we study effects of known or small errors-in-variables on such samplings. Throughout the paper, various simulations are also presented.
32 citations
TL;DR: In this article, an asymptotic expansion for the mean integrated squared error (MISE) of non-linear wavelet-based density estimators with randomly censored data is provided.
27 citations
TL;DR: In this article, the authors used the orthogonal series estimator corresponding to spherical harmonics for density estimation on the 2-sphere, S2, using the Fourier transform instead of truncating the empirical density.
18 citations
1 Jan 2003
TL;DR: It is shown, that PDE is a very good estimate for clusters of Gaussian structure and the robustness of the method is tested with respect to cluster overlap, number of clusters, different variances in different clusters and application to high dimensional data.
Abstract: Pareto Density Estimation (PDE) as defined in this work is a method for the estimation of probability density functions using hyperspheres. The radius of the hyperspheres is derived from optimizing information while minimizing set size. It is shown, that PDE is a very good estimate for clusters of Gaussian structure. The robustness of the method is tested with respect to cluster overlap, number of clusters, different variances in different clusters and application to high dimensional data. For high dimensional data PDE is found to be appropriate for the purpose of cluster analysis. The method is tested successfully on a difficult high dimensional real world problem: stock picking in falling markets.
16 citations
TL;DR: In this article, Paparoditis et al. proposed a new goodness-of-fit test for time series models based on spectral density estimation, which is based on the distance between a kernel estimator of the ratio of the true and hypothesized spectral density and the expected value of the estimator under the null and provides a quantification of how well the parametric density fits the sample spectral density.
Abstract: In a recent paper, Paparoditis [Scand. J. Statist. 27 (2000) 143] proposed a new goodness-of fit test for time series models based on spectral density estimation. The test statistic is based on the distance between a kernel estimator of the ratio of the true and the hypothesized spectral density and the expected value of the estimator under the null and provides a quantification of how well the parametric density fits the sample spectral density. In this paper, we give a detailed asymptotic analysis of the corresponding procedure under fixed alternatives.
TL;DR: In this article, it was shown that the Lp-distance between the kernel density estimators of the residuals and errors in the first-order autoregressive models is small.
20 Aug 2003
TL;DR: In this article, the authors proposed a nonparametric density estimation method that is computationally fast, capable of detecting density discontinuities and singularities at a very high resolution, spatially adaptive, and offers near minimax convergence rates for broad classes of densities including Besov spaces.
Abstract: The nonparametric density estimation method proposed in this paper is computationally fast, capable of detecting density discontinuities and singularities at a very high resolution, spatially adaptive, and offers near minimax convergence rates for broad classes of densities including Besov spaces. At the heart of this new method lie multiscale signal decompositions based on piecewise-polynomial functions and penalized likelihood estimation. Upper bounds on the estimation error are derived using an information-theoretic risk bound based on squared Hellinger loss. The method and theory share many of the desirable features associated with wavelet-based density estimators, but also offers several advantages including guaranteed non-negativity, bounds on the L1 error, small-sample quantification of the estimation errors, and additional flexibility and adaptability. In particular, the method proposed here can adapt the degrees as well as the locations of the polynomial pieces. For a certain class of densities, the error of the variable degree estimator converges at nearly the parametric rate. Experimental results demonstrate the advantages of the new approach compared to traditional density estimators and wavelet-based estimators.
TL;DR: Schucany et al. as mentioned in this paper proposed a kernel estimator which reduces the bias to the fourth power of the bandwidth, while the variance of the estimator increases only by at most a moderate constant factor.
Abstract: Kernel methods are very popular in nonparametric density estimation. In this article we suggest a simple estimator which reduces the bias to the fourth power of the bandwidth, while the variance of the estimator increases only by at most a moderate constant factor. Our proposal turns out to be a fourth order kernel estimator and may be regarded as a new version of the generalized jackknifing approach (Schucany W. R., Sommers, J. P. (1977). Improvement of Kernal type estimators. Journal of the American Statistical Association 72:420–423.) applied to kernel density estimation.
TL;DR: This paper establishes the asymptotic normality of a multistage kernel estimator of such quantities, by showing that under some regularity conditions on the underlying density function and on the kernels used on the multistages estimation procedure, theMultistagekernel estimator with at least one step of estimation is asymPTotically equivalent in probability to the kernel estimators with associated optimal bandwidth.
TL;DR: In this paper, the authors provide nonparametric estimators that nearly equal the MLE estimates for the marginal densities while being close to the kernel non-parametric density estimates for joint density estimates.
Abstract: When data analysts have multivariate data, often they have partial knowledge about the form of the marginal densities, but frequently they have little information about the bivariate and higher dimensional densities. This article provides nonparametric estimators that nearly equal the MLE estimates for the marginal densities while being close to the kernel nonparametric density estimates for the joint density estimates, provided that the assumption about the marginal densities is correct. The motivation for this article came from recollections of a 15-year-old conversation with Ingram Olkin where the problem at hand was how to model multivariate data with fixed marginals but with a flexible and rich multivariate structure.
TL;DR: In this article, the authors consider semiparametric asymmetric kernel density estimators when the unknown density has support on [0, ∞] and provide a unifying framework which contains asymmetric kernels versions of several semi-parametric estimation models.
Abstract: We consider semiparametric asymmetric kernel density estimators when the unknown density has support on [0, ∞). We provide a unifying framework which contains asymmetric kernel versions of several semiparametric density estimators considered previously in the literature. This framework allows us to use popular parametric models in a nonparametric fashion and yields estimators which are robust to misspecification. We further develop a specification test to determine if a density belongs to a particular parametric family. The proposed estimators outperform rival non- and semiparametric estimators in finite samples and are simple to implement. We provide applications to loss data from a large Swiss health insurer and Brazilian income data.
9 Nov 2003
TL;DR: A new model based approach for estimating a discrete probability density function, based on multirate dsp theory, is proposed, which yields an unbiased pdf estimate with small variance, which is guaranteed to have a smaller estimation error than the histogram.
Abstract: For many decades, the problem of estimating a pdf based on measurements has been of interest to many researchers. Even though much work has been done in the area of pdf estimation, most of it was focused on the continuous case. In this paper, we propose a new model based approach for estimating a discrete probability density function. This approach is based on multirate dsp theory, and it has several advantages over the traditional histogram method. It is shown that this method yields an unbiased pdf estimate with small variance, which is guaranteed to have a smaller estimation error than the histogram. Simulation results are given, which show the merit of the proposed method.
6 Apr 2003
TL;DR: It is found that varying an empirical bandwidth across resamples is largely unnecessary and thus, the computational burden is greatly reduced while maintaining estimation accuracy.
Abstract: We address the problem of confidence interval estimation of spectral densities using the bootstrap. Of special interest is the choice of the kernel global bandwidth. First, we investigate resampling based techniques for the choice of the bandwidth. We then address the question of whether the accuracy of the distributional bootstrap estimation is influenced by using the resample version, rather than the sample version of an empirical bandwidth. Aligned with recent results on non-parametric probability density estimation, we found that varying an empirical bandwidth across resamples is largely unnecessary and thus, the computational burden is greatly reduced while maintaining estimation accuracy.
TL;DR: Graphical representation of the family of estimated densities in three dimensional space provide a better explanation of the long‐term trends in temperature distribution of both series.
Abstract: Nonparametric density estimates attempt to reconstruct the probability density from which a random sample has come, using the sample values and as few assumptions as possible about the density. These methods are smoothing operations on the sample distribution. Methods of kernel estimates represent one of the most effective nonparametric methods. These methods are simple to understand, easy to implement and they have very good mathematical properties. We employed the automatic procedure for the selection of the bandwidth, the kernel and the order of the kernel. This procedure is used for analysis of air temperature fluctuations for series of Central England and Prague-Klementinum in the periods 1661-2000 and 1771-2000, respectively. Graphical representation of the family of estimated densities in three dimensional space provide a better explanation of the long-term trends in temperature distribution of both series.
1 Apr 2003
TL;DR: A pure data-driven method for density estimation, which provides good results even for small samples, which does not involve any problems or uncertainties as e.g. bandwidth selection for kernel density estimates.
Abstract: Based on an FQ-System for continuous unimodal distributions, which was introduced by Scheffner (1998), we propose a pure data-driven method for density estimation, which provides good results even for small samples. This procedure does not involve any problems or uncertainties as e.g. bandwidth selection for kernel density estimates.
TL;DR: It is found that the histograms perform better than kernel based methods when used as descriptors for CBIR applications and the experiments show that using over-smoothed bandwidth gives better retrieval performance.
Abstract: Color is widely used for content-based image retrieval. In these applications the color properties of an image are characterized by the probability distribution of the colors in the image. These probability distributions are very often estimated by histograms although the histograms have many drawbacks compared to other estimators such as kernel density methods.
In this paper we investigate whether using kernel density estimators instead of histograms could give better descriptors of color images. Experiments using these descriptors to estimate the parameters of the underlying color distribution and in color based image retrieval (CBIR) applications were carried out in which the MPEG7 database of 5466 color images with 50 standard queries are used as the benchmark. Noisy images are also generated and put into the CBIR application to test the robustness of the descriptors against the noise. The results of our experiments show that good density estimators are not necessarily good descriptors for CBIR applications. We found that the histograms perform better than kernel based methods when used as descriptors for CBIR applications.
In the second part of the paper, optimal values of important parameters in the construction of these descriptors, particularly the smoothing parameters or the bandwidth of the estimators, are discussed. Our experiments show that using over-smoothed bandwidth gives better retrieval performance.
1 Jan 2003
TL;DR: In this paper, nonparametric kernel density estimates were used for testing equality of distributions and cross-validated choice of bandwidth was used in the kernel density estimation, which was developed by resampling method, called the bootstrap.
Abstract: Hypothesis testing for the equality of two distributions were considered. Nonparametric kernel density estimates were used for testing equality of distributions. Cross-validatory choice of bandwidth was used in the kernel density estimation. Sampling distribution of considered test statistic were developed by resampling method, called the bootstrap. Small sample Monte Carlo simulation were conducted. Empirical power of considered tests were compared for variety distributions.
TL;DR: In this note non-parametric estimates of the length-biased probability density function and related reliability measures are presented andEqualities and bounds for the error of kernel estimators used for the estimation of length- biased probability densities are obtained.
TL;DR: In this article, a recipe for the construction of a multivariate probability density function from a normalized symmetric univariate function using a distance metric methodology is developed, and a method to perform multivariate integration in a polar coordinate system in an arbitary number of dimensions is described.
Abstract: [enter Abstract Body]A recipe for the construction of a multivariate probability density function from a normalized symmetric univariate function using a distance metric methodology is developed. A method to perform multivariate integration in a polar coordinate system in an arbitary number of dimensions is described. This method is then used to compute a constant that normalizes the multivariate p.d.f. constructed using the recipe specified here. The use of the Kolmogorov test, as applied to the univariate distribution of the square of the metric distance, for distributional identification is described. A summary of the general properties of the constructed p.d.f. is given including the computation of: the principal moments; Mardia's multivariate kurtosis measure; and, the characteristic and moment generatic functions. Some elements of maximum likelihood estimation are explored.
TL;DR: In this paper, a nonparametric method for the estimation of effective doses by kernel smoothing is proposed, which is based on the asymptotic properties of the proposed kernel estimator of dose response curves.
Abstract: A nonparametric method for the estimation of effective doses by kernel smoothing is proposed. The estimator of the dose and its asymptotic confidence interval are given. The estimation is based on the asymptotic properties of the proposed kernel estimator of dose response curves. The proposed method is compared with methods based on other kernel estimators and the parametric method via a simulation study, and is illustrated by applications to real data sets.
Proceedings Article•
2 Jun 2003TL;DR: A new method for density estimation based on Mercer kernels is presented, modifying the standard EM algorithm for mixtures of Gaussians to infer the parameters of the density.
Abstract: We present a new method for density estimation based on Mercer kernels. The density estimate can be understood as the density induced on a data manifold by a mixture of Gaussians fit in a feature space. As is usual, the feature space and data manifold are defined with any suitable positive-definite kernel function. We modify the standard EM algorithm for mixtures of Gaussians to infer the parameters of the density. One benefit of the approach is it's conceptual simplicity, and uniform applicability over many different types of data. Preliminary results are presented for a number of simple problems.