Multimodal visual information fusion aims to integrate the multi-sensor data into a single image which contains more complementary information and less redundant features. However the complementary information is hard to extract, especially for infrared and visible images which contain big similarity gap between these two modalities. The common cross attention modules only consider the correlation, on the contrary, image fusion tasks need focus on complementarity (uncorrelation). Hence, in this paper, a novel cross attention mechanism (CAM) is proposed to enhance the complementary information. Furthermore, a two-stage training strategy based fusion scheme is presented to generate the fused images. For the first stage, two auto-encoder networks with same architecture are trained for each modality. Then, with the fixed encoders, the CAM and a decoder are trained in the second stage. With the trained CAM, features extracted from two modalities are integrated into one fused feature in which the complementary information is enhanced and the redundant features are reduced. Finally, the fused image can be generated by the trained decoder. The experimental results illustrate that our proposed fusion method obtains the SOTA fusion performance compared with the existing fusion networks. The codes of our fusion method will be available soon. 

CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach

Emotions are a critical aspect of daily life and serve a crucial role in human decision-making, planning, reasoning, and other mental states. As a result, they are considered a significant factor in human interactions. Human emotions can be identified through various sources, such as facial expressions, speech, behavior (gesture/position), or physiological signals. The use of physiological signals can enhance the objectivity and reliability of emotion detection. Compared with peripheral physiological signals, electroencephalogram (EEG) recordings are directly generated by the central nervous system and are closely related to human emotions. EEG signals have the great spatial resolution that facilitates the evaluation of brain functions, making them a popular modality in emotion recognition studies. Emotion recognition using EEG signals presents several challenges, including signal variability due to electrode positioning, individual differences in signal morphology, and lack of a universal standard for EEG signal processing. Moreover, identifying the appropriate features for emotion recognition from EEG data requires further research. Finally, there is a need to develop more robust artificial intelligence (AI) including conventional machine learning (ML) and deep learning (DL) methods to handle the complex and diverse EEG signals associated with emotional states. This paper examines the application of DL techniques in emotion recognition from EEG signals and provides a detailed discussion of relevant articles. The paper explores the significant challenges in emotion recognition using EEG signals, highlights the potential of DL techniques in addressing these challenges, and suggests the scope for future research in emotion recognition using DL techniques. The paper concludes with a summary of its findings. 

Emotion recognition in EEG signals using deep learning methods: A review

AT-GAN: A generative adversarial network with attention and transition for infrared and visible image fusion

The varying speed can cause the significant data distribution shift of bearings, making it difficult for deep learning-based bearing fault diagnosis models to ensure good generalization. Domain adaptation methods have been developed to address the domain shifts, while they struggle with the class-invariant features extraction under variable speed. Accordingly, a time–frequency supervised contrastive learning framework (TF-SupCon) is proposed for unsupervised cross-speed fault diagnosis of bearing. TF-SupCon adopts a pre-training-downstream task framework that aims to extract speed immune class-invariant features. During the pre-training phase, the physical consistency of the time-domain information and the frequency-domain information of bearing signal, a general feature applicable to different speed conditions, is learned in a targeted manner through supervised contrastive learning. Additionally, a K-Nearest Neighbor (KNN) algorithm based on cosine distance is designed to assign pseudo-labels to unlabeled target domain data, enabling cross-domain supervised contrastive pre-training. In the downstream task, unsupervised cross-domain fault diagnosis is performed using a KNN classifier and the speed immune time–frequency features by the trained encoders. It is worth noting that the same metric is maintained throughout both the pre-training phase and the downstream task, ensuring organic connection between the two stages. The superiority of TF-SupCon was demonstrated through a variety of fault diagnosis experiments conducted on both public and self-collected datasets. Lastly, the distances between time-domain and frequency-domain features of different categories were studied to verify the physical consistency between the features. 

Time-frequency supervised contrastive learning via pseudo-labeling: An unsupervised domain adaptation network for rolling bearing fault diagnosis under time-varying speeds

Deep network-based maximum correlated kurtosis deconvolution: A novel deep deconvolution for bearing fault diagnosis

In general, the goal of the existing infrared and visible image fusion (IVIF) methods is to make the fused image contain both the high-contrast regions of the infrared image and the texture details of the visible image. However, this definition would lead the fusion image losing information from the visible image in high-contrast areas. For this problem, this article proposed a decoupling network-based IVIF method (DNFusion), which utilizes the decoupled maps to design additional constraints on the network to force the network to retain the saliency information of the source image effectively. The current definition of image fusion is satisfied while effectively maintaining the saliency objective of the source images. Specifically, the feature interaction module (FIM) inside effectively facilitates the information exchange within the encoder and improves the utilization of complementary information. Also, a hybrid loss function constructed with weight fidelity loss, gradient loss, and decoupling loss ensures the fusion image to be generated to effectively preserve the source image’s texture details and luminance information. The qualitative and quantitative comparison of extensive experiments demonstrates that our model can generate a fused image containing saliency objects and clear details of the source images, and the method we proposed has a better performance than other state-of-the-art (SOTA) methods.

Infrared and Visible Image Fusion via Decoupling Network

The rapid developments in mini‐hardware manufacturing and wireless network communications have enabled the Internet of Medical Things (IoMT) to provide continuous healthcare services over the Internet. Federated learning (FL) combined with blockchain technology has been a popular way to resolve privacy‐preserving data sharing in IoMT‐based wireless body area networks (WBANs), on the other side, communication payloads become much heavier than traditional healthcare sensor network, because central server should aggregate the model updates and orchestrate the training tasks. The high latency will lead to FL's low system efficiency. However, the existing studies on FL mainly focus on the system design and algorithm optimization, which ignore a critical problem of data transmission in the FL system. To improve the communication performance, we proposed a two‐tier scheduling algorithm in which a full‐duplex (FD) multiple access based scheduling algorithm is employed to improve channel utility and network throughput, and decrease the delay in tier II. A deep reinforcement learning (DRL) framework is used to generate the FD links between hubs and access points (APs) which jointly considers the channel state, fairness, and delay. Therefore, the DRL‐based FD Link Scheduling (R‐FDLS) algorithm is proposed. When the traffic volume is different or in various distribution scenarios, the evaluation results demonstrate that the proposed algorithm significantly improves the network communication quality, as well as has obvious advantages compared to several baselines.

Deep reinforcement learning‐based full‐duplex link scheduling in federated learning‐based computing for IoMT

A reinforcement learning based full-duplex multi- user multi-input multi-output (RL-MUFD) Multiple Access Control protocol is proposed. RL-MUFD is a centralized-control MAC protocol where an FD access point (AP) selects multiple simultaneous uplink and downlink streams from half-duplex users and bypasses channel estimation solely based on the geographic locations and the payloads of users. The objective of dynamic link scheduling is to find a policy that maximizes the expected long-term reward that maintains a trade-off among channel utilization, throughput, delay and traffic balance. This problem can be formulated as a partially observable Markov decision process (POMDP). The RL structure based on Proximal Policy Optimization is developed to solve the suggested formulation. We trained the proposed algorithm using the actor-critic neural network architecture over randomly deployed networks. Simulation results show that in most scenarios, RL-MUFD outperforms other benchmark algorithms in terms of overall performance. In the mobile scene, RL-MUFD achieves a delayed gain of 22.69%, 45.86% and 26.73% compared with Random, Max-rate and FD- MUMAC protocols, respectively.

Reinforcement Learning Based Full Duplex Multi-user MIMO MAC Protocol (RL-MUFD)

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ya Li

Author Tools

Chat about Author

Papers

Infrared and Visible Image Fusion via Decoupling Network

Deep reinforcement learning‐based full‐duplex link scheduling in federated learning‐based computing for IoMT

Reinforcement Learning Based Full Duplex Multi-user MIMO MAC Protocol (RL-MUFD)