Robust partially linear models for automatic structure discovery

TL;DR: In this article , the authors proposed a robust linear and nonlinear discovery algorithm (RLAND) by integrating the modal regression and partially linear models (PLMs), which can mitigate the influence of noise in structure discovery.

Abstract: Partially linear models (PLMs), rooted in the combination of linear and nonlinear approximation, are recognized to be capable of modeling complex data. Indeed, the performance of PLMs depends heavily on the choice of model structure, such as which covariates have linear or nonlinear effects on the response. Nevertheless, most existing PLMs are limited to the mean regression, resulting in sensitivity to non-Gaussian noises, such as skewed noise and heavy-tailed noise. In order to mitigate the influence of noise in structure discovery, this paper proposes a Robust Linear And Nonlinear Discovery algorithm (RLAND) by integrating the modal regression and PLMs. Statistical analysis on generalization bound and structure discovery consistency are established to characterize its learning theory foundations. Computation analysis illustrates that the RLAND can be efficiently realized by half quadratic optimization and the quadratic programming. Empirical evaluations on simulation and real-world data validate the competitive performance of the proposed method.