Multi-response Regression for Block-missing Multi-modal Data without Imputation

Question

1. How can multi-modal block-wise missing data be handled in multi-response regression models?

2. What is the problem setup in multi-response linear regression model?

3. What is the proposed Multi-DISCOM method in the context of multi-response linear regression models?

4. What is the computational algorithm for solving optimization problem (2.5)?

Accepted Answer

In this paper, a multi-response regression model for block-wise missing data is proposed. The method includes two steps: estimating the covariance and cross-covariance matrices using all available observations without imputation, and using a penalized approach to estimate the sparse regression coefficient matrix and the precision matrix of the error terms simultaneously. This method allows missing values in both responses and predictors and considers correlations among responses. It also handles the case where no subject has complete observations. Theoretical properties, simulation studies, and a multi-modal ADNI data example are presented to demonstrate the effectiveness of the proposed method.

Accepted Answer

The problem setup in multi-response linear regression model involves Y = XB * + E, where B * = (b jk ) R pxq is an unknown p x q parameter matrix, Y = (y 1, . . ., y n ) is the n x q response matrix, X = (x 1, . . ., x n ) is the n x p design matrix, and E = (1, . . ., n) is the n x q error matrix. The predictors are assumed to come from multiple modalities with p k predictors in the k-th modality. X has block-missing values, and elements of Y can also be missing. The errors i = (i1, . . ., iq) for i = 1, . . ., n are i.i.d. realizations from a random vector with zero mean and covariance matrix S = (s EE ij ) R qxq. The notation used includes S dxd + for sets of d x d symmetric positive-definite matrices, tr(C) for the trace of a square matrix C, diag(C) for the diagonal matrix of C, and various norms and eigenvalues for matrices A.

Accepted Answer

The proposed Multi-DISCOM method is a two-step weighted LASSO approach that aims to improve the predictive performance of multi-response linear regression models. It incorporates the correlations between responses by solving an optimization problem that simultaneously estimates B* and C*. The method is represented by the equation (B, C) = arg min CS qxq + ,B tr C SYY + CB SXX B - 2CB SXY +l B B 1 + l C C 1 - log det C. When l C is large enough, the method reduces to the separate LASSO. For univariate response regression problems, it reduces to the DISCOM algorithm, and for cases with no missing entries, it reduces to the sparse conditional Gaussian graphical model. The proposed method has been shown to have better estimation performance than the two-step weighted LASSO and the separate LASSO, as demonstrated by the toy example in Section 2.2.1.

Accepted Answer

The computational algorithm for solving optimization problem (2.5) involves an alternating minimization method. Starting with initial points (B0, C0), at each iteration, the algorithm solves two subproblems: Bt = arg min B tr Ct-1 SY Y + Ct-1 B SXX B - 2 Ct-1 B SXY + l B B 1 and Ct = arg min CS qxq + tr C SY Y + C B t-1 SXX Bt-1 - 2C B t-1 SXY + l C C 1 - log det C. The quadratic subproblem for Bt is solved using the coordinate descent algorithm, while the graphical lasso method is used for Ct. The algorithm iterates until a convergence threshold is reached, updating Bt and Ct accordingly.

Accepted Answer

In numerical studies, the Multi-DISCOM method demonstrates superior performance compared to other methods. It achieves smaller mean squared error (MSE) and estimation errors across various settings. The method excels particularly when there are large correlations between different responses. Additionally, the Multi-DISCOM method outperforms the Lasso method using imputed data, which may suffer from randomness in imputing block-missing data. The method also shows more advantage when the signal-to-noise ratio is smaller, as it effectively considers the precision matrix for estimation. Overall, the Multi-DISCOM method proves to be robust and efficient in handling numerical studies related to S, signal-to-noise ratio, and error distribution.

Accepted Answer

The Multi-DISCOM is applied to the Alzheimer's Disease Neuroimaging Initiative (ADNI) study to identify biological markers and neuropsychological assessments for MCI and early AD progression. It compares with existing approaches, revealing that while some features are frequently selected by Multi-DISCOM and known to be highly correlated with AD and MCI, the DISCOM rarely selects these features. This suggests that Multi-DISCOM may offer a more effective method for feature selection in this context. The selection frequency of each feature is defined as the number of times it is selected in 30 replications, as shown in Figure 2. The Multi-DISCOM's ability to consistently select relevant features makes it a promising approach for analyzing ADNI data and potentially improving the understanding of MCI and early AD progression.

Accepted Answer

The proposed estimation method in the penalized framework involves a joint estimation method with entry-wise 1 regularization using block-missing multi-modal predictors. It estimates the covariance matrix of the predictors through a linear combination of variance estimates, intra-modality covariance matrix estimates, and cross-modality covariance matrix estimates. This estimator can be positive semidefinite and more accurate than the sample covariance matrix. The method also includes a penalized estimator for sparse coefficient estimation in optimal linear prediction. Theoretical studies on estimation and feature selection consistency are established, and extensive simulation studies show promising performance on estimation, prediction, and model selection for block-missing multi-modal data. The Multi-DISCOM model is applied to the ADNI dataset, demonstrating good prediction power and meaningful interpretation.

Multi-response Regression for Block-missing Multi-modal Data without Imputation

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. How can multi-modal block-wise missing data be handled in multi-response regression models?

2. What is the problem setup in multi-response linear regression model?

3. What is the proposed Multi-DISCOM method in the context of multi-response linear regression models?

4. What is the computational algorithm for solving optimization problem (2.5)?

5. How does the Multi-DISCOM method perform in numerical studies related to S, signal-to-noise ratio, and error distribution?

6. How does Multi-DISCOM compare to existing approaches in ADNI study?

7. What is the proposed estimation method in the penalized framework?

Citations

Interpretable Automated Machine Learning for Asset Pricing in U.S. Capital Markets

The ACM Multimedia 2025 Grand Challenge of Multimodal Conversational Aspect-based Sentiment Analysis

References

Regression Shrinkage and Selection via the Lasso

Sparse inverse covariance estimation with the graphical lasso

Sparse inverse covariance estimation with the graphical lasso.

The Alzheimer's disease neuroimaging initiative

On Model Selection Consistency of Lasso

Related Papers (5)

Four Methods of Handling Missing Data with the 1984 General Social Survey.

The Robustness of Regression and Substitution by Mean Methods in Handling Missing Values.

Missing data imputation using weighted of regime switching mean and regression

Performance Comparison of Imputation Methods for Mixed Data Missing at Random with Small and Large Sample Data Set with Different Variability

Imputation methods for missing response values in the three parts of a central composite design with two factors