Kernel Regression Utilizing External Information as Constraints

Question

1. What is the focus of nonparametric kernel regression in the context of using external datasets for internal data analysis?

2. How can k-fold cross-validation be applied for bandwidth selection?

3. How to handle bias in kernel estimator?

4. How does heterogeneity affect internal and external data populations?

Accepted Answer

Nonparametric kernel regression, a well-established approach, does not require any assumption on the regression function between Y and U, except for some smoothness conditions. It focuses on utilizing summary information from external datasets to improve the estimation of the regression function. In the context of using external datasets for internal data analysis, nonparametric kernel regression aims to make use of external information to obtain better fitted regression values at each observed U in the internal dataset. This approach helps overcome the curse of dimensionality for kernel-type methods and enhances the performance of nonparametric regression methods. Theoretical and empirical studies on the performance of constrained kernel (CK) regression demonstrate that it can improve the standard kernel method without using external information, especially when the sample size of the external dataset is at least comparable to the sample size of the internal dataset. The proposed CK regression method involves a two-step process, where the first step applies a constrained optimization procedure to obtain fitted regression values, and the second step applies standard kernel regression to estimate the entire regression function. The performance of CK regression is measured using the asymptotic mean integrated square error (AMISE), and the results show that it can significantly enhance the estimation accuracy compared to the standard kernel method.

Accepted Answer

K-fold cross-validation can be applied for bandwidth selection by partitioning the internal dataset into approximately equal-sized subsets (G1, ..., Gk). For each subset, the estimator u(-j)CK(Ui) is calculated using bandwidths l and b, excluding the data {(Yi, Ui), iGj}. The chosen bandwidths (l, b) that minimize the CV(l, b) = k∑(iGj) {u(-j)CK(Ui) - Yi}^2 are selected. This method ensures a reasonable range of bandwidth selection and provides an effective way to optimize bandwidths for a given sample size n.

Accepted Answer

The main technical difficulty in handling bias in the kernel estimator of u(u) is estimating the asymptotic bias B K (u) for standard kernel estimation and B CK (u) for constrained kernel estimation. These biases are not zero unless c = 0, and c > 0 leads to the best convergence rate. Bias estimation is challenging, but the idea of under smoothing can be applied. Under smoothing involves choosing bandwidths smaller than those selected by CV for confidence intervals. For the constrained kernel estimation, bandwidths c l l and c b b are used in the first and second stages, respectively. A confidence interval [u CK (u) - z a V 1/2 CK (u), u CK (u) + z a V 1/2 CK (u)] is set for u(u), where V CK is the variance estimator and s 2 CK (u) is the variance estimator for the standard kernel. When s 2 (u) does not depend on u, a simplified estimator is used. The performance of this confidence interval is examined through simulation in Section 3.2.

Accepted Answer

Heterogeneity in populations for internal and external data can impact the estimation of regression functions. When populations differ, observed internal data (Y, U, R) and external data (Y, X) are considered separately. The goal is to estimate the regression function for internal data population, assuming EQUATION EQUATION R i {b g g(X i ) - u i }g(X i ) = 0. To address heterogeneity, condition E(Y | X, R = 1) = E(Y | X, R = 0) and X R is required. This condition allows for the estimation of b g in (2.15) and E{b g g(X)g(X) |R = 1}. External information, such as E[E{Y |X, R = 1}g(X) |R = 1] = E{u 1 (U )g(X) |R = 1}, is available to extend the method. Additionally, generalized estimating equations (GEE) can be used to generalize the approach to different types of external data, such as logistic regression for discrete responses. The constraint N i=1 R i H( b, u i , X i ) = 0 can be applied for GEE summary-level information.

Accepted Answer

The simulation results presented in this section compare the performance of the proposed CK estimator (2.7) with the standard kernel estimator (2.2) without using external information. Two cases are considered: (i) bounded covariates, where X and Z are uniformly distributed on [-1, 1], and (ii) normal covariates, where (X, Z) follows a bivariate normal distribution with specified means, variances, and correlation. The response Y is normal, and four different models (M1-M4) are used to generate u(X, Z). The internal and external datasets are independently sampled from the same population, with sizes n = 200 and m = 1,000, respectively. Additionally, a total of N = 1,200 data are generated, with R indicating internal or external data based on a probability distribution. The simulation results provide insights into the performance of the CK estimator compared to the kernel estimator in different scenarios.

Accepted Answer

The Mean Integrated Square Error (MISE) is a measure used to evaluate the performance of kernel estimators in simulation studies. It is calculated by simulation with S replications, where MISE = 1/S * S * s=1 * 1/T * T * {u(s)1(U s,t) - u1(U s,t)}^2. In this formula, {U s,t : t = 1, ..., T} represents the test data for each simulation replication s, and the simulation is repeated independently for s = 1, ..., S. The estimator u1 is defined by (2.13), and u(s)1 is an estimator of u1 using a method described previously based on internal and external data, independent of test data. The MISE is used to compare the performance of different kernel estimators and assess their efficiency in terms of mean integrated square error. The lower the MISE value, the better the performance of the kernel estimator.

Accepted Answer

Confidence intervals with bounded covariates show good performance in terms of coverage probability (CP). Table 3 demonstrates that all confidence intervals perform well in this scenario. The intervals based on the CK method have shorter lengths compared to those based on the standard kernel without external information. This indicates that the CK method is more efficient in estimating the true u(u) value. However, for normally distributed covariates, the CP may not be very good in some cases, suggesting that the asymptotic theory has not fully kicked in. Despite this, the CK interval generally has a shorter length than the interval based on the standard kernel.

Accepted Answer

InO2 and etO2 are considered as two components of Z in kernel regression. Since the internal dataset has a sample size of 32, a linear combination of heart rate, respiratory rate, and blood oxygen saturation is used as a one-dimensional covariate X. The coefficients of this linear combination are derived from the first eigenvector of the SAVE algorithm, which provides more than 94% variability. Therefore, the kernel regression uses a three-dimensional covariate U, which includes inO2 and etO2 as components of Z. The external dataset MIMIC-III, which does not have inO2 and etO2 as covariates, is used to compare the results and validate the findings.

Accepted Answer

The CK method, proposed in Section 2, is designed for low-dimensional covariate U, where p is small. If p is not small, dimension reduction techniques should be applied before using CK or kernel methods. For instance, in a single index model, the SIR technique can be used to obtain a consistent and asymptotically normal estimator. Other dimension reduction techniques, such as those by Cook and Weisberg (1991) and Shao et al. (2007), can also be applied. When dealing with high-dimensional X, component-wise constraints can be used instead of constraint (2.5) to mitigate the impact of dimension. The second-order kernel is adopted to achieve a convergence rate of n-2/(4+p). Alternative nonparametric smoothing techniques, like the local polynomial Statistica Sinica, can also be used to achieve a convergence rate of n-d/(2d+p), where d >= 2. The CK method's results can be extended to scenarios with multiple external datasets, requiring component-wise constraints and potentially combining different methods for varying populations.

Kernel Regression Utilizing External Information as Constraints

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What is the focus of nonparametric kernel regression in the context of using external datasets for internal data analysis?

2. How can k-fold cross-validation be applied for bandwidth selection?

3. How to handle bias in kernel estimator?

4. How does heterogeneity affect internal and external data populations?

5. How do simulation results compare CK estimator and kernel estimator?

6. How does the Mean Integrated Square Error (MISE) measure the performance of kernel estimators in simulation studies?

7. How do confidence intervals perform with bounded covariates?

8. How are inO2 and etO2 considered in kernel regression?

9. How does the CK method address the curse of dimensionality?

Citations

Kernel regression utilizing heterogeneous datasets

References

MIMIC-III, a freely accessible critical care database

A Distribution-Free Theory of Nonparametric Regression

All of Nonparametric Statistics

On Directional Regression for Dimension Reduction

A Model-Calibration Approach to Using Complete Auxiliary Information From Survey Data

Related Papers (5)

The Hilbert Kernel Regression Estimate

Landsat-8 Image Restoration Based on Kernel Density Regression

Adaptive warped kernel estimation for nonparametric regression with circular responses

Recursive weighted kernel regression for semi-supervised soft-sensing modeling of fed-batch processes

SLiKER: Sparse loss induced kernel ensemble regression