Abstract: Exploring and identifying structure is even more important for multivariate data than univariate data, given the difficulties in graphically presenting multivariate data and the comparative lack of parametric models to represent it. Unfortunately, such exploration is also inherently more difficult.

Abstract: Local likelihood was introduced by Tibshirani and Hastie as a method of smoothing by local polynomials in non-Gaussian regression models. In this paper an extension of these methods to density estimation is discussed, and comparison with other methods of density estimation presented. The local likelihood method has particularly strong advantages over kernel methods when estimating tails of densities and in multivariate settings. Suppose constraints are incorporated in a simple manner. Asymptotic properties of the estimate are discussed. A method for computing the estimate is outlined. C code to implement the estimation procedure described in this paper, together with S interfaces for graphical display of results, are available at: http://cm.bell-labs.com/stat/project/locfit/index.html

Abstract: Bickel and Ritov's estimator is a quite intricate expression based on kernel estimators of the density. In this paper, we propose an alternative and somehow simpler method of estimation based on orthogonal projections. This

Abstract: Recently, the authors introduced the concept of the socalled Local Discriminant Basis (LDB) for signal and image classification problems [6], [17, Chap. 4], [19], [20]. This method first decomposes available training signals in a time-frequency dictionary (also known as a dictionary of orthonormal bases ) which is a large collection of the basis functions (such as wavelet packets and local trigonometric functions) localized both in time and in frequency. Then, signal energies at the basis coordinates are accumulated for each signal class separately to form a time-frequency energy distribution per class. Based on the differences among these energy distributions (measured by a certain “distance” functional), a complete orthonormal basis called LDB, which “can see” the distinguishing signal features among signal classes, is selected from the dictionary. After the basis is determined, expansion coefficients in the most important several coordinates (features) are fed into a traditional classifier such as linear discriminant analysis (LDA) or classification tree (CT). Finally, the corresponding coefficients of test signals are computed and fed to the classifier to predict their classes. This LDB concept has been increasingly popular and applied to a variety of classification problems including geophysical acoustic waveform classification [18], radar signal classification [11], and classification of neuron firing patterns of monkeys to different stimuli [22]. Through these studies, we have found that the criterion used in the original LDB algorithm—the one based on the differences of the time-frequency energy distributions among signal classes—is not always the best one to use. Consider an artificial problem as follows. Suppose one class of signals consists of a single basis function in a time-frequency dictionary with its amplitude and they are embedded in white Gaussian noise (WGN) with zero mean and unit variance. The other class of signals consists of the same basis function but with its amplitude and again they are embedded in the same WGN process. Then their time-frequency energy distributions are identical. Therefore, we cannot select the right basis function as a discriminator. This simple counterexample suggests that we should also consider the differences of the distributions of the expansion coefficients in each basis coordinate. In this example, all coordinates except the one corresponding to the single basis function have the same Gaussian distribution. The probability density function (pdf) of the projection of input signals onto this one basis function should reveal twin peaks around . In this paper we propose a new LDB algorithm based on the differences among coordinate-wise pdfs as a basis selection criterion and we explain similarities and differences among the original LDB algorithm and the new LDB algorithm.

Abstract: We develop a fully Bayesian solution to the density estimation problem. Smoothness of the estimates f is incorporated through the integral formulation f(x) = ∫ dx′ф(x′) K(x,x′) involving an appropriately smooth kernel function K. The analysis involves integration over the underlying space of densities ф. The key to this approach lies in properly setting up a measure on this space consistent with passage to the continuum limit of continuous x. With this done, a flat prior suffices to complete a well-posed definition of the problem.

Abstract: The problem of estimating the marginal density of a linear process by kernel methods is considered. Under general conditions, kernel density estimators are shown to be asymptotically normal. Their limiting covariance matrix is computed. We also find the optimal bandwidth in the sense that it asymptotically minimizes the mean square error of the estimators. The assumptions involved are easily verifiable.

Abstract: In this article we construct a kernel estimate of the probability density function from bivariate data that have been randomly censored. We study the large-sample properties of the proposed estimator using a strong approximation result. We establish consistency and asymptotic normality and give a convenient representation of the kernel density estimator. Simulation studies show that the proposed procedure gives a good estimate of the true density function even when the sample size is moderate. We discuss various issues about implementation of the estimator, including bandwidth selection and boundary effects. The procedure can be generalized to higher dimensional variables in a straightforward manner.

Abstract: For the purpose of nonparametric density estimation, a prior distribution is constructed on the space of stepwise constant density functions, not necessarily of bounded support. In particular, the sequence of heights is conditionally distributed a priorias a Dirichlet process on the integers, given a bidimensional mixing parameter. Such a mixing parameter is composed of a bin width and a starting point which are, in turn, assigned an arbitrary marginal prior. Proper Bayesian estimates of the density are obtained. They are not histograms, but they share common features with the histogram and other kernel based estimators. They also incorporate prior information, like a prior guess for the density or bounds for its support, which may be particularly appealing for small sample situations, where usual density estimation methods are not satisfactory. The estimates are computable by simple numerical methods, as opposed to other nonparametric Bayesian density estimators proposed in the literature, which display ...

Abstract: Smooth nonparametric kernel density and regression estimators are studied when the data is strongly dependent. In particular, we derive Central (and Noncentral) Limit Theorems for the kernel density estimator of a multivariate Gaussian process and infinite-order moving average of an independent identically distributed process, as well as its consistency for other types of data, such as nonlinear functions of a Gaussian process. Also, Central (and Noncentral) Limit Theorems of the nonparametric kernel regression estimators are studied. One important and surprising characteristic found is that its asymptotic variance does not depend on the point at which the regression function is estimated and also it is found that their asymptotic properties are the same whether or not the regressors are strongly dependent. A Monte Carlo experiment is reported to assess the behaviour of the estimators in finite samples.

Abstract: censored observation, and if 6, = 1, X, denotes a survival time, which is the variable of interest In this paper, we apply the martingale method for counting processes to study asymptotic properties for the kernel estimator of the density function of the survival times We also derive an asymptotic expression for the mean integrated square error of the kernel density estimator, which can be used to obtain an asymptotically optimal bandwidth

Abstract: A method for the detection of regime shifts in univariate time series is investigated in this paper by combining nonparametric regression and density estimation techniques with prediction tests for structural changes. An application of the method to US quarterly ex-post real interest rates is presented.

Abstract: A method of obtaining weak or strong invariance principles for the centered and normalized deconvoluting kernel estimator is presented. It is shown that these approximations, under certain smoothness constraints, achieve the best global rates of convergence.

Abstract: We propose to model the unknown distribution F of a sequence of independent real-valued random variables by partitioning the real line into intervals I 0ors (MRFPs). We argue and illustrate that many commonly-expressed prior opinions about the shape and form of F can be expressed as statements about the joint distribution of neighboring p i 's, leading to simple MRFP expressions for prior beliefs that are awkward to express in other models. In particular, we will show how to model beliefs about log concavity, unimodality, and monotonicity. The posterior distributions of the p i 's in our models (and hence the approximate predictive distributions for subsequent observations) are readily computed using Markov chain Monte-Carlo methods.

Abstract: We propose a method for estimating mean squared error and bandwidth in the windowed spectral density estimation of a stationary Gaussian process, and also provide a method for estimating the second order derivative of the spectral density function. The asymptotic properties and the convergence rates of the estimators are given.

Abstract: A fundamental concept in the analysis of univariate data is the probability density function. Let X be a random variable that has probability density function f(x). The density function describes the distribution of X and allows probabilities to be determined using the relation $$P(a < X < b) = \int\limits_a^b {f(u)du.} $$

Abstract: In the classical kernel approximation of a probability density function, a fixed zero-mean smooth approximation of the Dirac delta function is translated to each sample value of the random variable to form a family of density functions. The sample values are the means of the density functions in the corresponding family. The kernel approximation is the average over all members of this family. In certain applications, there may be too few sample values to give a good approximation. In this paper, we propose an approach that utilizes the sample values as the variances of a family of density functions. This approach works particularly well with a Gaussian mixture density.

Abstract: New classes of efficient estimators are proposed for functionals of the distribution density function or of the spectral density function. Bibliography: 14 titles.

Abstract: Unlike most nonlinear system identification tasks, which focus on determining the unknown system functions parametrically or nonparametrically to be used in future prediction of the output values given specific inputs, many statistical signal processing applications require the estimation of the corresponding conditional density function so that a probabilistic decision can be made. In this paper, we propose a new method to estimate the multivariate conditional density, f(m/x), a density over the output space m conditioned on any given input x. In particular, we are interested in cases where the number of available training data, points is relatively sparse within x space. We start from a priori considerations and establish certain desirable characteristics in kernel functions for conditional density estimation. We find that Gaussian kernels with expanding covariances, expanding as we move away from the data point of the kernel, satisfy these a priori considerations. We combine these expanding Gaussian kernels (EGK) according to Bayesian techniques. We compare the EGK with standard Gaussian kernel (SDK) methods, and find that EGK avoids multimodality, has diminishing confidence levels farther from training points, performs better asymptotically and performs better with respect to the Kullback-Leibler criteria.

Abstract: The problem of estimating the marginal density of a linear process by kernel methods is considered. Under general conditions, kernel density estimators are shown to be asymptotically normal. Their limiting covariance matrix is computed. We also find the optimal bandwidth in the sense that it asymptotically minimizes the mean square error of the estimators. The assumptions involved are easily verifiable.

Abstract: Kernel-type estimators of the multivariate density of stationary random fields indexed by multidimensional lattice points space are investigated. Sufficient conditions for kernel estimators to converge in L 1 are obtained. The results are applicable to a large class of spatial processes.