(2001). Empirical Processes in M-Estimation. Journal of the American Statistical Association: Vol. 96, No. 454, pp. 779-780.

Empirical Processes in M-Estimation

Multitask learning is a popular approach to training high-performing neural networks with improved generalization. In this paper, we propose a background class to achieve improved generalization at a lower computation compared to multitask learning to help researchers and organizations with limited computation power. We also present a methodology for selecting background images and discuss potential future improvements. We apply our approach to several datasets and achieved improved generalization with much lower computation. We also investigate class activation mappings (CAMs) of the trained model and observed the tendency towards looking at a bigger picture in a few class classification problems with the proposed model training methodology. Applying transformer with the proposed background class, we receive state-of-the-art (SOTA) performance on STL-10, CIFAR-10, CIFAR-100, Oxford-102, Caltech-101, and CINIC-10 datasets. Example scripts are available in the 'CAM' folder of the following GitHub Repository: github.com/dipuk0506/UQ

Reduction of Class Activation Uncertainty with Background Information

Logarithmic sample bounds for Sample Average Approximation with capacity- or budget-constraints

Abstract Complex systems are generating more and more high-dimensional data for which tensor analysis showed promising results by capturing complex correlation structures of data. Such data is often distributed among various sites creating challenges for developing data-driven models. Specifically, data privacy and security concerns have been exacerbated in recent years and drove the demand to store and analyze data at the edge of networks rather than sharing it with a centralized server. Federated learning frameworks have been introduced as a solution to these concerns. These frameworks allow local clients to learn local models and collaborate with others to develop a more generalizable aggregated model while handling data privacy issues. In this article, we propose a federated generalized scalar-on-tensor regression framework where multiple local tensor models are learned at the edge, and their parameters are shared with and updated by an aggregator. Experiments on synthetic data sets and two real-world data sets from agriculture and manufacturing domains show the superiority of our approach over several benchmarks.

Federated generalized scalar-on-tensor regression

In recent years, diversified measurements reflect the system dynamics from a more comprehensive perspective in system modeling and analysis, such as scalars, waveform signals, images, and structured point clouds. To handle such multimodal structured high-dimensional (SHD) data, combining a large amount of data from multiple sites is necessary (i) to reduce the inherent population bias from a single site and (ii) to increase the model accuracy. However, impeded by data management policies and storage costs, data could not be easily shared or directly exchanged among different sites. Instead of simplifying or facilitating the data query process, we propose a federated multiple tensor-on-tensor regression (FedMTOT) framework to train the individual system model locally using (i) its own data and (ii) data features (not data itself) from other sites. Specifically, federated computation is executed based on alternating direction method of multipliers (ADMM) to satisfy data-sharing requirements, while the individual model at each site can still benefit from feature knowledge from other sites to improve its own model accuracy. Finally, two simulations and two case studies validate the superiority of the proposed FedMTOT framework. 

Federated Multiple Tensor-on-Tensor Regression (FedMTOT) for Multimodal Data under Data-Sharing Constraints

In recent years, high-dimensional data, such as waveform signals and images have become ubiquitous. This type of data is often represented by multiway arrays or tensors. Several statistical models, including tensor regression, have been developed for such tensor data. However, these models are sensitive to the presence of arbitrary outliers within the tensors. To address the issue, this article proposes a Robust Tensor-On-Tensor (RTOT) regression approach, which has the capability of modeling high-dimensional data when the data is corrupted by outliers. Through several simulations and case studies, we evaluate the performance of the proposed method. The results reveal the advantage of the RTOT over some benchmarks in the literature in terms of estimation error. A Python implementation is available at https://github.com/Reisi-Lab/RTOT.git. 

Robust Tensor-On-Tensor Regression for Multidimensional Data Modeling

This paper concerns a high-dimensional stochastic programming problem of minimizing a function of expected cost with a matrix argument. To this problem, one of the most widely applied solution paradigms is the sample average approximation (SAA), which uses the average cost over sampled scenarios as a surrogate to approximate the expected cost. Traditional SAA theories require the sample size to grow rapidly when the problem dimensionality increases. Indeed, for a problem of optimizing over a $p$-by-$p$ matrix, the sample complexity of the SAA is given by $\tilde O(1)\cdot \frac{p^2}{\epsilon^2}\cdot{polylog}(\frac{1}{\epsilon})$ to achieve an $\epsilon$-suboptimality gap, for some poly-logarithmic function ${polylog}(\,\cdot\,)$ and some quantity $\tilde O(1)$ independent of dimensionality $p$ and sample size $n$. In contrast, this paper considers a regularized SAA (RSAA) with a low-rankness-inducing penalty. We demonstrate that the sample complexity of RSAA is $\tilde O(1)\cdot \frac{p}{\epsilon^3}\cdot {polylog}(p,\,\frac{1}{\epsilon})$, which is almost linear in $p$ and thus indicates a substantially lower dependence on dimensionality. Therefore, RSAA can be more advantageous than SAA especially for larger scale and higher dimensional problems. Due to the close correspondence between stochastic programming and statistical learning, our results also indicate that high-dimensional low-rank matrix recovery is possible generally beyond a linear model, even if the common assumption of restricted strong convexity is completely absent.

Regularized Sample Average Approximation for High-Dimensional Stochastic Optimization Under Low-Rankness

While a number of practical methods for training quantized DL models have been presented in the literature, there exists a critical gap in the theoretical generalizability results for such approaches. Although empirical evidence often suggests a high tolerance of DL architectures to variations of training procedures, existing theoretical generalization analyses are often contingent on the specific designs of training algorithms, e.g., in stochastic gradient descent (SGD). This specialization makes such generalizability results inapplicable to the case of quantized DL models. In view of this critical vacuum, this paper provides several almost-algorithm-independent results to ensure the generalizability of a quantized neural network at different levels of optimality. These results include the characterizations of a computable, quantized local solution that ensures the generalization performance and an algorithm that is provably convergent to such a local solution. • A novel generalizability theory for quantized deep neural nets trained globally. • The first generalization error bound for tractable local solutions of quantized DL. • Proposing a provably effective and efficient algorithm for quantized DL models. 

Training generalizable quantized deep neural nets

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Hung Yi Lee

Author Tools

Chat about Author

Papers

Robust Tensor-On-Tensor Regression for Multidimensional Data Modeling

Regularized Sample Average Approximation for High-Dimensional Stochastic Optimization Under Low-Rankness

Training generalizable quantized deep neural nets