Abstract: Mendelian randomization (MR) is a method of exploiting genetic variation to unbiasedly estimate a causal effect in presence of unmeasured confounding. MR is being widely used in epidemiology and other related areas of population science. In this paper, we study statistical inference in the increasingly popular two-sample summary-data MR design. We show a linear model for the observed associations approximately holds in a wide variety of settings when all the genetic variants satisfy the exclusion restriction assumption, or in genetic terms, when there is no pleiotropy. In this scenario, we derive a maximum profile likelihood estimator with provable consistency and asymptotic normality. However, through analyzing real datasets, we find strong evidence of both systematic and idiosyncratic pleiotropy in MR, echoing the omnigenic model of complex traits that is recently proposed in genetics. We model the systematic pleiotropy by a random effects model, where no genetic variant satisfies the exclusion restriction condition exactly. In this case, we propose a consistent and asymptotically normal estimator by adjusting the profile score. We then tackle the idiosyncratic pleiotropy by robustifying the adjusted profile score. We demonstrate the robustness and efficiency of the proposed methods using several simulated and real datasets.

Abstract: This study assesses whether the accrual-generating process is adequately described by a linear model with respect to a range of underlying determinants examined by prior literature. We document substantial departures from linearity across the distributions of accrual determinants, including measures of size, performance, and growth. To incorporate non-linear relations, we employ a recently developed multivariate matching approach (entropy balancing) to adjust for determinants in place of relying on a linear model. Entropy balancing identifies weights for the control sample to equalize the distribution of determinants across treatment and control samples. In simulations drawing random samples from deciles where a linear model displays poor fit, we find that entropy balancing significantly improves accrual model specification by reducing coefficient bias relative to linear and propensity-score matched models. Consistent with entropy balancing retaining sufficient power, we find that its estimates detect seeded accrual manipulations and explain variation in accruals around equity issuances.

Abstract: The recent growth of data science is partly fueled by the ever-growing amount of data and the joint important developments in statistical modeling, with new and powerful models and frameworks becoming accessible to users. Although there exist some generic functions to obtain model summaries and parameters, many package-specific modeling functions do not provide such methods to allow users to access such valuable information.

Abstract: Inference, prediction, and control of complex dynamical systems from time series is important in many areas, including financial markets, power grid management, climate and weather modeling, or molecular dynamics. The analysis of such highly nonlinear dynamical systems is facilitated by the fact that we can often find a (generally nonlinear) transformation of the system coordinates to features in which the dynamics can be excellently approximated by a linear Markovian model. Moreover, the large number of system variables often change collectively on large time- and length-scales, facilitating a low-dimensional analysis in feature space. In this paper, we introduce a variational approach for Markov processes (VAMP) that allows us to find optimal feature mappings and optimal Markovian models of the dynamics from given time series data. The key insight is that the best linear model can be obtained from the top singular components of the Koopman operator. This leads to the definition of a family of score functions called VAMP-r which can be calculated from data, and can be employed to optimize a Markovian model. In addition, based on the relationship between the variational scores and approximation errors of Koopman operators, we propose a new VAMP-E score, which can be applied to cross-validation for hyper-parameter optimization and model selection in VAMP. VAMP is valid for both reversible and nonreversible processes and for stationary and nonstationary processes or realizations.

Abstract: Recently, deep learning has unlocked unprecedented success in various domains, especially using images, text, and speech. However, deep learning is only beneficial if the data have nonlinear relationships and if they are exploitable at available sample sizes. We systematically profiled the performance of deep, kernel, and linear models as a function of sample size on UKBiobank brain images against established machine learning references. On MNIST and Zalando Fashion, prediction accuracy consistently improves when escalating from linear models to shallow-nonlinear models, and further improves with deep-nonlinear models. In contrast, using structural or functional brain scans, simple linear models perform on par with more complex, highly parameterized models in age/sex prediction across increasing sample sizes. In sum, linear models keep improving as the sample size approaches ~10,000 subjects. Yet, nonlinearities for predicting common phenotypes from typical brain scans remain largely inaccessible to the examined kernel and deep learning methods. Schulz et al. systematically benchmark performance scaling with increasingly sophisticated prediction algorithms and with increasing sample size in reference machine-learning and biomedical datasets. Complicated nonlinear intervariable relationships remain largely inaccessible for predicting key phenotypes from typical brain scans.

Abstract: Introduction and Aims The COVID-19 pandemic originated from the city of Wuhan of China has highly affected the health, socio-economic and financial matters of the different countries of the world. India is one of the countries which is affected by the disease and thousands of people on daily basis are getting infected. In this paper, an analysis of daily statistics of people affected by the disease are taken into account to predict the next days trend in the active cases in Odisha as well as India. Material and methods A valid global data set is collected from the WHO daily statistics and correlation among the total confirmed, active, deceased, positive cases are stated in this paper. Regression model such as Linear and Multiple Linear Regression techniques are applied to the data set to visualize the trend of the affected cases. Results Here a comparison of Linear Regression and Multiple Linear Regression model is performed where the score of the model R 2 tends to be 0.99 and 1.0 which indicates a strong prediction model to forecast the next coming days active cases. Using the Multiple Linear Regression model as on July month, the forecast value of 52,290 active cases are predicted towards the next month of 15th August in India and 9,358 active cases in Odisha if situation continues like this way. Conclusion These models acquired remarkable accuracy in COVID-19 recognition. A strong correlation factor determines the relationship among the dependent (active) with the independent variables (positive, deceased, recovered).

Abstract: Epidemiologic research often involves meta-analyses of proportions. Conventional two-step methods first transform each study's proportion and subsequently perform a meta-analysis on the transformed scale. They suffer from several important limitations: the log and logit transformations impractically treat within-study variances as fixed, known values and require ad hoc corrections for zero counts; the results from arcsine-based transformations may lack interpretability. Generalized linear mixed models (GLMMs) have been recommended in meta-analyses as a one-step approach to fully accounting for within-study uncertainties. However, they are seldom used in current practice to synthesize proportions. This article summarizes various methods for meta-analyses of proportions, illustrates their implementations, and explores their performance using real and simulated datasets. In general, GLMMs led to smaller biases and mean squared errors and higher coverage probabilities than two-step methods. Many software programs are readily available to implement these methods.

Abstract: Dynamic multiobjective optimization problems (DMOPs) challenge multiobjective evolutionary algorithms (MOEAs) because those problems change rapidly over time. The class of DMOPs whose objective functions change over time steps, in ways that exhibit some hidden patterns has gained much attention. Their predictability indicates that the problem exhibits some correlations between solutions obtained in sequential time periods. Most of the current approaches use linear models or similar strategies to describe the correlations between historical solutions obtained, and predict the new solutions in the following time period as an initial population from which the MOEA can begin searching in order to improve its efficiency. However, nonlinear correlations between historical solutions and current solutions are more common in practice, and a linear model may not be suitable for the nonlinear case. In this paper, we present a support vector regression (SVR)-based predictor to generate the initial population for the MOEA in the new environment. The basic idea of this predictor is to map the historical solutions into a high-dimensional feature space via a nonlinear mapping, and to do linear regression in this space. SVR is used to implement this process. We incorporate this predictor into the MOEA based on decomposition (MOEA/D) to construct a novel algorithm for solving the aforementioned class of DMOPs. Comprehensive experiments have shown the effectiveness and competitiveness of our proposed predictor, comparing with the state-of-the-art methods.

Abstract: The coefficient of determination R2 quantifies the amount of variance explained by regression coefficients in a linear model. It can be seen as the fixed-effects complement to the repeatability R (intra-class correlation) for the variance explained by random effects and thus as a tool for variance decomposition. The R2 of a model can be further partitioned into the variance explained by a particular predictor or a combination of predictors using semi-partial (part) R2 and structure coefficients, but this is rarely done due to a lack of software implementing these statistics. Here, we introduce partR2, an R package that quantifies part R2 for fixed effect predictors based on (generalized) linear mixed-effect model fits. The package iteratively removes predictors of interest and monitors the change in R2 as a measure of the amount of variance explained uniquely by a particular predictor or a set of predictors. partR2 also estimates structure coefficients as the correlation between a predictor and fitted values, which provide an estimate of the total contribution of a fixed effect to the overall prediction, independent of other predictors. Structure coefficients are converted to the total variance explained by a predictor, termed ‘inclusive’ R2, as the square of the structure coefficients times total R2. Furthermore, the package reports beta weights (standardized regression coefficients). Finally, partR2 implements parametric bootstrapping to quantify confidence intervals for each estimate. We illustrate the use of partR2 with real example datasets for Gaussian and binomials GLMMs and discuss interactions, which pose a specific challenge for partitioning the explained variance among predictors.

Abstract: A continuing mystery in understanding the empirical success of deep neural networks is their ability to achieve zero training error and generalize well, even when the training data is noisy and there are more parameters than data points. We investigate this overparameterized regime in linear regression, where all solutions that minimize training error interpolate the data, including noise. We lower-bound the fundamental generalization (mean-squared) error of any interpolating solution in the presence of noise, and show that this bound decays to zero with the number of features. Thus, overparameterization can be beneficial in ensuring harmless interpolation of noise. We discuss two root causes for poor generalization that are complementary in nature – signal “bleeding” into a large number of alias features, and overfitting of noise by parsimonious feature selectors. For the sparse linear model with noise, we provide a hybrid interpolating scheme that mitigates both these issues and achieves order-optimal MSE over all possible interpolating solutions.

Abstract: We study generalised linear regression and classification for a synthetically generated dataset encompassing different problems of interest, such as learning with random features, neural networks in the lazy training regime, and the hidden manifold model. We consider the high-dimensional regime and using the replica method from statistical physics, we provide a closed-form expression for the asymptotic generalisation performance in these problems, valid in both the under- and over-parametrised regimes and for a broad choice of generalised linear model loss functions. In particular, we show how to obtain analytically the so-called double descent behaviour for logistic regression with a peak at the interpolation threshold, we illustrate the superiority of orthogonal against random Gaussian projections in learning with random features, and discuss the role played by correlations in the data generated by the hidden manifold model. Beyond the interest in these particular problems, the theoretical formalism introduced in this manuscript provides a path to further extensions to more complex tasks.

Abstract: We propose leave-out estimators of quadratic forms designed for the study of linear models with unrestricted heteroscedasticity. Applications include analysis of variance and tests of linear restrictions in models with many regressors. An approximation algorithm is provided that enables accurate computation of the estimator in very large datasets. We study the large sample properties of our estimator allowing the number of regressors to grow in proportion to the number of observations. Consistency is established in a variety of settings where plug-in methods and estimators predicated on homoscedasticity exhibit first-order biases. For quadratic forms of increasing rank, the limiting distribution can be represented by a linear combination of normal and non-central χ2 random variables, with normality ensuing under strong identification. Standard error estimators are proposed that enable tests of linear restrictions and the construction of uniformly valid confidence intervals for quadratic forms of interest. We find in Italian social security records that leave-out estimates of a variance decomposition in a two-way fixed effects model of wage determination yield substantially different conclusions regarding the relative contribution of workers, firms, and worker-firm sorting to wage inequality than conventional methods. Monte Carlo exercises corroborate the accuracy of our asymptotic approximations, with clear evidence of non-normality emerging when worker mobility between blocks of firms is limited.

Abstract: In recent decades, various conventional techniques have been formulated around the world to evaluate the overall water quality (WQ) at particular locations. In the present study, back propagation neural network (BPNN) and adaptive neuro-fuzzy inference system (ANFIS), support vector regression (SVR), and one multilinear regression (MLR) are considered for the prediction of water quality index (WQI) at three stations, namely Nizamuddin, Palla, and Udi (Chambal), across the Yamuna River, India. The nonlinear ensemble technique was proposed using the neural network ensemble (NNE) approach to improve the performance accuracy of the single models. The observed WQ parameters were provided by the Central Pollution Control Board (CPCB) including dissolved oxygen (DO), pH, biological oxygen demand (BOD), ammonia (NH3), temperature (T), and WQI. The performance of the models was evaluated by various statistical indices. The obtained results indicated the feasibility of the developed data intelligence models for predicting the WQI at the three stations with the superior modelling results of the NNE. The results also showed that the minimum values for root mean square (RMS) varied between 0.1213 and 0.4107, 0.003 and 0.0367, and 0.002 and 0.0272 for Nizamuddin, Palla, and Udi (Chambal), respectively. ANFIS-M3, BPNN-M4, and BPNN-M3 improved the performance with regard to an absolute error by 41%, 4%, and 3%, over other models for Nizamuddin, Palla, and Udi (Chambal) stations, respectively. The predictive comparison demonstrated that NNE proved to be effective and can therefore serve as a reliable prediction approach. The inferences of this paper would be of interest to policymakers in terms of WQ for establishing sustainable management strategies of water resources.

Abstract: The classical linear multi-factor stock selection model is widely used for long-term stock price trend prediction. However, the stock market is chaotic, complex, and dynamic, for which reasons the linear model assumption may be unreasonable, and it is more meaningful to construct a better-integrated stock selection model based on different feature selection and nonlinear stock price trend prediction methods. In this paper, the features are selected by various feature selection algorithms, and the parameters of the machine learning-based stock price trend prediction models are set through time-sliding window cross-validation based on 8-year data of Chinese A-share market. Through the analysis of different integrated models, the model performs best when the random forest algorithm is used for both feature selection and stock price trend prediction. Based on the random forest algorithm, a long-short portfolio is constructed to validate the effectiveness of the best model.

Abstract: Intensive research in the field of mathematical modeling of hydraulic servo systems has shown that their mathematical models have many important details which cannot be included in the model. Due to impossibility of direct measurement or calculation of dimensions of certain components, leakage coefficients or friction coefficients, it was supposed that parameters of the hydraulic servo system are random. On the other side, it has been well known that the hydraulic servo system can be approximated by a linear model with time-varying parameters. An estimation of states and time-varying parameters of linear state-space models is of practical importance for fault diagnosis and fault-tolerant control. Previous works on this topic consider estimation in Gaussian noise environment, but not in the presence of outliers. The known fact is that the measurements have inconsistent observations with the largest part of the observation population (outliers). They can significantly make worse the properties of linearly recursive algorithms which are designed to work in the presence of Gaussian noises. This paper proposes the strategy of parameter–state robust estimation of linear state-space models in the presence of all possible faults and non-Gaussian noises. Because of its good features in robust filtering, Masreliez–Martin filter represents a cornerstone for realization of the robust algorithm. The good features of the proposed robust algorithm to identification of the hydraulic servo system are illustrated by intensive simulations.

Abstract: Linearization is often used for control design of nonlinear systems but what degree of a linearization is sufficient for the controller design is always a question. Furthermore, most of the existing linearization methods aim to develop a completely linear model without retaining any nonlinearity and thus the unmodeled dynamics unavoidably exists due to omitted higher order terms. In this article, a local compact form dynamic linearization (local-CFDL) is developed at first to transform the original nonlinear nonaffine system into an affine structure consisting of both an unknown residual nonlinear time-varying term and a linearly parametric term affine to the control input. A discrete-time extended state observer (DESO) is introduced to estimate the unknown residual nonlinear time-varying term as a new extended state. Then, a local-CFDL-based DESO-model-free adaptive control (MFAC) is proposed where the estimation of DESO is incorporated to compensate for the disturbances and uncertainties. Furthermore, a local partial-form dynamic linearization (local-PFDL) is also presented using multi-lag inputs and partial derivatives. And, a corresponding local-PFDL-based DESO-MFAC is proposed utilizing additional control information to improve control performance. The two proposed methods are both data-driven and do not require any explicit model information. Theoretical analysis shows the robust convergence of the proposed methods in the presence of disturbances. Simulations verify the effectiveness of the proposed method and show that the local-PFDL-based DESO-MFAC outperforms the local-CFDL-based one owing to the use of additional control information.

Abstract: Best-subset selection aims to find a small subset of predictors, so that the resulting linear model is expected to have the most desirable prediction accuracy. It is not only important and imperative in regression analysis but also has far-reaching applications in every facet of research, including computer science and medicine. We introduce a polynomial algorithm, which, under mild conditions, solves the problem. This algorithm exploits the idea of sequencing and splicing to reach a stable solution in finite steps when the sparsity level of the model is fixed but unknown. We define an information criterion that helps the algorithm select the true sparsity level with a high probability. We show that when the algorithm produces a stable optimal solution, that solution is the oracle estimator of the true parameters with probability one. We also demonstrate the power of the algorithm in several numerical studies.

Abstract: Frames, shells, and hybrid structures with early damages, such as early cracks, often behave as extremely weak nonlinear systems, among which the nonlinearity is difficult to be detected, especially if the system response is affected by the noise. To avoid these damages becoming catastrophic failures, developing effective incipient damages detection methods is important. The nonlinear output frequency response functions (NOFRFs) and associated indexes can be considered as one kind of the prospective detection tools, which are usually determined from the established nonlinear autoregressive with exogenous inputs (NARX) model. However, the hyperparameters in the NARX model are difficult to be determined so that the identification accuracy cannot be guaranteed. Therefore, it is important to develop more accurate methods to estimate the NOFRFs and their associated indicators for damage detection. Motivated by the powerful learning abilities of convolutional neural networks (CNN) and long short-term memory (LSTM) networks, a novel method based on NOFRFs and the CNN-LSTM model for detecting the early damages in structures is proposed. By applying the beat excitation, the response of the structure is divided into two components, where the approximately linear component is used to estimate the frequency characteristic of the linear component by the classical linear model and the nonlinear component is used to establish the CNN-LSTM model. By calculating the responses of the two models, the NOFRFs and associated indexes can be accurately estimated, and then the early damage can be detected. Simulation and experimental studies verify the potential and effectiveness of the novel method proposed in this article.

Abstract: Water quality index is a measure of water quality at a certain location and over a period of time. High value indicates that the water is unsafe for drinking and inadequate in quality to meet the designated uses. Most of the classical models are unreliable producing unpromising forecasting results. This study presents Artificial Intelligence (AI) techniques and a Multi Linear Regression (MLR) as the classical linear model for estimating the Water Quality Index (WQI) of Palla station of Yamuna river, India. Full-scale data of the river were used in validating the models. Performance measures such as Mean Square Error (MSE), Root Mean Squared Error (RMSE) and Determination Coefficient (DC) were utilized in evaluating the accuracy and performance of the models. The obtained result depicted the superiority of AI models over the MLR model. The results also indicated that, the best model of both ANN and ANFIS proved high improvement in performance accuracy over MLR up to 10% in the verification phase. The difference between ANN and ANFIS accuracy is negligible due to a slight increment in performance accuracy indicating that both ANN and ANFIS could serve as reliable models for the estimation of WQI .

Abstract: In this paper, we propose, discuss, and validate an online Nonlinear Model Predictive Control (NMPC) method for multi-rotor aerial systems with arbitrarily positioned and oriented rotors which simultaneously addresses the local reference trajectory planning and tracking problems. This work brings into question some common modeling and control design choices that are typically adopted to guarantee robustness and reliability but which may severely limit the attainable performance. Unlike most of state of the art works, the proposed method takes advantages of a unified nonlinear model which aims to describe the whole robot dynamics by explicitly including a realistic physical description of the actuator dynamics and limitations. As a matter of fact, our solution does not resort to common simplifications such as: (1) linear model approximation, (2) cascaded control paradigm used to decouple the translational and the rotational dynamics of the rigid body, (3) use of low-level reactive trackers for the stabilization of the internal loop, and (4) unconstrained optimization resolution or use of fictitious constraints. More in detail, we consider as control inputs the derivatives of the propeller forces and propose a novel method to suitably identify the actuator limitations by leveraging experimental data. Differently from previous approaches, the constraints of the optimization problem are defined only by the real physics of the actuators, avoiding conservative – and often not physical – input/state saturations which are present, e.g., in cascaded approaches. The control algorithm is implemented using a state-of-the-art Real Time Iteration (RTI) scheme with partial sensitivity update method. The performances of the control system are finally validated by means of real-time simulations and in real experiments, with a large spectrum of heterogeneous multi-rotor systems: an under-actuated quadrotor, a fully-actuated hexarotor, a multi-rotor with orientable propellers, and a multi-rotor with an unexpected rotor failure. To the best of our knowledge, this is the first time that a predictive controller framework with all the valuable aforementioned features is presented and extensively validated in real-time experiments and simulations.

Abstract: Multiple imputation (MI) is increasingly popular for handling multivariate missing data. Two general approaches are available in standard computer packages: MI based on the posterior distribution of incomplete variables under a multivariate (joint) model, and fully conditional specification (FCS), which imputes missing values using univariate conditional distributions for each incomplete variable given all the others, cycling iteratively through the univariate imputation models. In the context of longitudinal or clustered data, it is not clear whether these approaches result in consistent estimates of regression coefficient and variance component parameters when the analysis model of interest is a linear mixed effects model (LMM) that includes both random intercepts and slopes with either covariates or both covariates and outcome contain missing information. In the current paper, we compared the performance of seven different MI methods for handling missing values in longitudinal and clustered data in the context of fitting LMMs with both random intercepts and slopes. We study the theoretical compatibility between specific imputation models fitted under each of these approaches and the LMM, and also conduct simulation studies in both the longitudinal and clustered data settings. Simulations were motivated by analyses of the association between body mass index (BMI) and quality of life (QoL) in the Longitudinal Study of Australian Children (LSAC). Our findings showed that the relative performance of MI methods vary according to whether the incomplete covariate has fixed or random effects and whether there is missingnesss in the outcome variable. We showed that compatible imputation and analysis models resulted in consistent estimation of both regression parameters and variance components via simulation. We illustrate our findings with the analysis of LSAC data.

Abstract: This letter deals with a very simple question: if we have grouped data with a binary-dependent variable and want to include fixed effects in the specification, can we meaningfully compare results using a linear model to those estimated with a logit? The reason to doubt such a comparison is that the linear specification appears to keep all observations, whereas the logit drops the groups where the dependent variable is either all zeros or all ones. This letter demonstrates that a linear specification averages the estimates for all the homogeneous outcome groups (which, by definition, all have slope coefficients of zero) with the slope coefficients for the groups with a mix of zeros and ones. The correct comparison of the linear to logit form is to only look at groups with some variation in the dependent variable. Researchers using the linear specification are urged to report results for all groups and for the subset of groups where the dependent variable varies. The interpretation of the difference between these two results depends upon assumptions which cannot be empirically assessed.

Abstract: Survival regression is used to estimate the relation between time-to-event and feature variables, and is important in application domains such as medicine, marketing, risk management and sales management. Nonlinear tree based machine learning algorithms as implemented in libraries such as XGBoost, scikit-learn, LightGBM, and CatBoost are often more accurate in practice than linear models. However, existing implementations of tree-based models have offered limited support for survival regression. In this work, we propose and implement loss functions for learning accelerated failure time (AFT) models in XGBoost, to increase the support for survival modeling for different kinds of label censoring. The AFT model assumes effects that directly accelerate or decelerate the survival time for different kinds of censored data sets. We demonstrate with real and simulated experiments the effectiveness of AFT in XGBoost with respect to a number of baselines, in two respects: generalization performance and training speed. Furthermore, we take advantage of the support for NVIDIA GPUs in XGBoost to achieve substantial speedup over multi-coreCPUs. To our knowledge, our work is the first implementation of AFT that utilizes the processing power of NVIDIA GPUs.