A robust operators’ cognitive workload recognition method based on denoising masked autoencoder

Although deep neural networks exhibit high performance on several specific tasks, they suffer from a severe issue of catastrophic forgetting when learning new tasks incrementally. Several incremental learning methods have been recently proposed to address this problem, some of which rely on storing data or constructing generative models to enhance the performance of incremental models. However, storing old data from previous tasks may be limited by memory or privacy issues, and generative models may often be unstable and inefficient during model training. In this study, we present a non-exemplar-based class-incremental learning method to mitigate catastrophic forgetting in incremental learning when saving old class samples is not possible, while distinguishing between old and new classes. We tackle the problem of representation bias and classifier bias that are inherent in class-incremental learning. Specifically, we introduce class enhancement, which enables the model to encounter more classes during training to acquire transferable and diverse representation features, mitigating representation bias. Furthermore, we use backtracking transfer, which employs semantic mapping to limit inter-class relationships by learning class-level semantic relationships across two tasks, to preserve discriminative old knowledge and prevent catastrophic forgetting due to classifier bias. We perform comprehensive experiments on three benchmark datasets that demonstrate the superiority of our proposed method over state-of-the-art methods. 

Augmented Transport Data-Free Class Incremental Learning

Self-supervised approaches for electroencephalography (EEG) representation learning face three specific challenges inherent to EEG data: (1) The low signal-to-noise ratio which challenges the quality of the representation learned, (2) The wide range of amplitudes from very small to relatively large due to factors such as the inter-subject variability, risks the models to be dominated by higher amplitude ranges, and (3) The absence of explicit segmentation in the continuous-valued sequences which can result in less informative representations. To address these challenges, we introduce \textit{EEG2Rep}, a self-prediction approach for self-supervised representation learning from EEG. Two core novel components of EEG2Rep are as follows: 1) Instead of learning to predict the masked input from raw EEG, EEG2Rep learns to predict masked input in latent representation space, and 2) Instead of conventional masking methods, EEG2Rep uses a new semantic subsequence preserving (SSP) method which provides informative masked inputs to guide EEG2Rep to generate rich semantic representations. In experiments on 6 diverse EEG tasks with subject variability, EEG2Rep significantly outperforms state-of-the-art methods. We show that our semantic subsequence preserving improves the existing masking methods in self-prediction literature and find that preserving 50\% of EEG recordings will result in the most accurate results on all 6 tasks on average. Finally, we show that EEG2Rep is robust to noise addressing a significant challenge that exists in EEG data. Models and code are available at:\url{https://github.com/Navidfoumani/EEG2Rep} 

pdf/eeg2rep-enhancing-self-supervised-eeg-representation-through-20d70tv9tf.pdf

EEG2Rep: Enhancing Self-supervised EEG Representation Through
  Informative Masked Inputs

Multivariate biosignals such as Electroencephalography (EEG) and Electrocardiography (ECG) are widely used for understanding various pathologies that affect the brain and cardiovascular activity. However, effectively modeling such signals is challenging due to their complex temporal patterns, long-range dependencies, and dynamically evolving structures. Traditional models, such as recurrent neural networks, convolutional neural networks, and Transformers, encounter challenges with long-term temporal modeling, scalability, and computational efficiency. In this work, we propose a novel model that integrates Mamba architecture, channel attention, and dynamic graph learning for efficient and adaptive long-range temporal modeling of biosignals. Specifically, We address the biosignals modeling problem through: (i) long-range temporal modeling using parallel Mamba layers that process both time and frequency domain representations; (ii) a low-cost channel attention mechanism that identifies discriminative sensor channels with minimal computational overhead; and (iii) dynamic graph structure learning that adapts graph representations over time to capture evolving spatial relationships in biosignal data. We validate the proposed approach on three benchmark datasets: TUSZ dataset (for EEG-based epileptic seizure detection), DOD-H dataset (for EEG-based sleep stage classification), and ICBEB dataset (for ECG-based cardio disease classification). The model achieves state-of-the-art performance on these datasets. Notably, ablation studies demonstrate the effectiveness of each proposed component.

pdf/adaptive-long-range-modeling-of-eeg-and-ecg-with-mamba-and-6kojspbe9xqm.pdf

Adaptive long-range modeling of EEG and ECG with Mamba and dynamic graph learning

Catastrophic forgetting is the central challenge in class incremental learning (CIL), namely adapting a model to new data often results in severe performance degradation on previous tasks or classes. The most successful methods to alleviate this forgetting require extensive replay of previously seen data, which is problematic when memory constraints or data legality concerns exist. In non-exemplar-based CIL setting, an important factor causing catastrophic forgetting is the presence of severe bias between the new and previously learned classes in both the feature extractor and classifier. To address this problem, this paper proposes a simple and effective solution to calibrate the bias in non-exemplar-based CIL. Specifically, we propose to maintain the decision boundaries of the previously learned classes by preserving a collection of pseudo-samples in the latent space. By maintaining the distinction and balance between old and new classes mitigates representation bias. Then, to further maintain fairness, we propose classifier aligning that corrects the bias in the FC layer, which help the model output more discriminative results within older classes. Experimental results on benchmark datasets show that our method is significantly superior to non-exemplar-based methods, with a 10% improvement in average incremental accuracy, and achieves comparable performance compared to exemplar-based methods. The ablation study further confirms the effectiveness of the method. 

Maintaining Distinction and Fairness in Data-Free Class Incremental Learning

Electroencephalogram (EEG) is a non-invasive technique to record bioelectrical signals. Integrating supervised deep learning techniques with EEG signals has recently facilitated automatic analysis across diverse EEG-based tasks. However, the label issues of EEG signals have constrained the development of EEG-based deep models. Obtaining EEG annotations is difficult that requires domain experts to guide collection and labeling, and the variability of EEG signals among different subjects causes significant label shifts. To solve the above challenges, self-supervised learning (SSL) has been proposed to extract representations from unlabeled samples through well-designed pretext tasks. This paper concentrates on integrating SSL frameworks with temporal EEG signals to achieve efficient representation and proposes a systematic review of the SSL for EEG signals. In this paper, 1) we introduce the concept and theory of self-supervised learning and typical SSL frameworks. 2) We provide a comprehensive review of SSL for EEG analysis, including taxonomy, methodology, and technique details of the existing EEG-based SSL frameworks, and discuss the difference between these methods. 3) We investigate the adaptation of the SSL approach to various downstream tasks, including the task description and related benchmark datasets. 4) Finally, we discuss the potential directions for future SSL-EEG research.

pdf/self-supervised-learning-for-electroencephalogram-a-w7of0g276i.pdf

Self-supervised Learning for Electroencephalogram: A Systematic Survey

Domain generalization (DG), which aims to learn a model that can generalize to an unseen target domain, has recently attracted increasing research interest. A major approach is to learn domain invariant representations to avoid greedily capturing all the correlations found in source domains caused by empirical risk minimization. Nevertheless, overly emphasizing learning of domain invariant representations might lead to learning overly-compressed domain invariant representations, causing confusion of different classes in a same domain. To address this limitation, we introduce a novel dynamic domain-weighted contrastive loss, which maximizes the subdomain differences between different classes especially those belonging to the same domain, while minimizing the average distance between the points of the convex hull of the aligned source domains. We propose Multi-source domain-adversarial generalization via dynamic domain-weighted Contrastive transfer learning (MsCtrl), a novel domain-adversarial generalization framework, which optimizes the distribution alignment of source and potential target subdomains in an adversarial manner under the “control” of the aforementioned contrastive loss. Extensive experiments based on real-world datasets demonstrate significant advantages of MsCtrl over existing state-of-the-art methods. 

Letting Go of Self-Domain Awareness: Multi-Source Domain-Adversarial Generalization via Dynamic Domain-Weighted Contrastive Transfer Learning

Domain adaptation can apply knowledge learned from the source domain to the target domain by reducing data distribution discrepancy inter domains. However, existing domain adaptation algorithms do not do as well on sensor datasets as on image datasets because of the neglect of intra data distribution discrepancy. The long time of collecting a raw data segment on sensors will lead to a shift with time, and the shift distribution will change with the variety of sensors and wearing positions, causing the time series distribution discrepancy intra and inter domains. To solve this problem, we design a new model, Time Series Adaptation Network (TSAN), and a new loss, Time series Contrastive Loss (TCL). TSAN uses a siamese network and “pack” the samples divided from the same segment into the network. Furthermore, TCL is defined as the similarity of “unpack” network output, which leads the model to learn time-independent features. In particular, TSAN can be used as a plug-in to combine with existing domain adaptation algorithms, so the intra and inter distribution discrepancies can be considered simultaneously. We conduct extensive experiments with eight existing domain adaptation algorithms on sensor-based cross domain human activity recognition (HAR) tasks, including three Routine Activity Recognition (RAR) datasets and four Parkinson's tremor Detection (PD) datasets. The results show that all the existing algorithms are improved by an average of 5.4% (RAR), 2.2% (PD) with TSAN. 

Time Series Adaptation Network for Sensor-Based Cross Domain Human Activity Recognition

Wearable-based human activity recognition (HAR) is commonly employed in real-world scenarios such as health monitoring, auxiliary diagnosis, etc. As implementing activity recognition is a daunting challenge in an open dynamic environment, incremental learning has become a common method to adapt to variable behavior patterns of users and create dynamic modeling in activity recognition. However, catastrophic forgetting is a significant challenge with incremental learning. This is contrary to our expectations of identifying new activity classes while remembering existing ones. To address this problem, we propose a knowledge importance-based class incremental learning method called KiCi and construct an incremental learning model based on the framework of self-iterative knowledge distillation for dynamic activity recognition. To eliminate the prediction bias of the teacher model on the old knowledge, we utilize the trained weights of previous incremental steps generated by the teacher model as the prior knowledge to obtain knowledge importance. Then use it to make the student model have a reasonable trade-off between old and new knowledge and mitigate catastrophic forgetting by avoiding negative transfer. We conduct extensive experiments on four public HAR datasets and our method consistently outperforms the existing state-of-the-art methods by a large margin. 

KiCi

DDIR: Domain-Disentangled Invariant Representation learning for tailored predictions

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Yuan Ma

Author Tools

Chat about Author