This paper describes the 6th Affective Behavior Analysis in-the-wild (ABAW) Competition, which is part of the respective Workshop held in conjunction with IEEE CVPR 2024. The 6th ABAW Competition addresses contemporary challenges in understanding human emotions and behaviors, crucial for the development of human-centered technologies. In more detail, the Competition focuses on affect related benchmarking tasks and comprises of five sub-challenges: i) Valence-Arousal Estimation (the target is to estimate two continuous affect dimensions, valence and arousal), ii) Expression Recognition (the target is to recognise between the mutually exclusive classes of the 7 basic expressions and 'other'), iii) Action Unit Detection (the target is to detect 12 action units), iv) Compound Expression Recognition (the target is to recognise between the 7 mutually exclusive compound expression classes), and v) Emotional Mimicry Intensity Estimation (the target is to estimate six continuous emotion dimensions). In the paper, we present these Challenges, describe their respective datasets and challenge protocols (we outline the evaluation metrics) and present the baseline systems as well as their obtained performance. More information for the Competition can be found in: https://affective-behavior-analysis-in-the-wild.github.io/6th. 

The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition

Facial expression recognition(FER) is an exceedingly challenging task in the field of computer vision, primarily due to variant head poses, occlusions and illumination conditions. Previous methods focus on extracting more facial features to improve the performance on the FER benchmarks. However, most of them ignore the weights of the key regions. To address the above problem, this paper proposes a vision Transformer with weight reallocation module termed as WR-Former. The weight reallocation module utilizes a hybrid multi-head self-attention and can dynamically iterate during the training process. It guides the model to reallocate tokens based on the importance of different facial regions. Besides, a simple yet effective data augmentation method is proposed to expand training samples and improve the robustness. Experimental results and visualizations demonstrate that our WR-Former outperforms the previous state-of-the-art methods and has the ability to better extract the features conveyed by the key facial regions.

WR-Former: Vision Transformer with Weight Reallocation Module for Robust Facial Expression Recognition

This paper presents our work to the Valence-Arousal Estimation Challenge of the 5th Affective Behavior Analysis in-the-wild (ABAW) competition. We explore the problems in this VA challenge from three aspects: 1) To obtain efficient and robust feature representations, we explore the role of multiple visual and video feature extractors; 2) Based on multimodal feature representations that fuse the visual and video information, we utilize four types of temporal encoders to capture the temporal context information in the video, including the LSTM, GRU, Transformer based encoder and a combined encoder of Transformer and LSTM; 3) five model ensemble strategies are used to combine multiple results with different model settings. Our system achieves the performance in Concordance Correlation Coefficients (CCC) of 0.6193 for valence, 0.6634 for arousal, and a mean CCC of 0.6414 on the test set, which demonstrates the effectiveness of our proposed method and ranks first place in the challenge. 

EVAEF: Ensemble Valence-Arousal Estimation Framework in the Wild

The in-the-wild affective behavior analysis has been an important study. In this paper, we submit our solutions for the 5th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW), which includes V-A Estimation, Facial Expression Classification and AU Detection Sub-challenges. We propose a Transformer Encoder with Multi-Head Attention framework to learn the distribution of both the spatial and temporal features. Besides, there are virious effective data augmentation strategies employed to alleviate the problems of sample imbalance during model training. The results fully demonstrate the effectiveness of our proposed model based on the Aff-Wild2 dataset.

pdf/spatial-temporal-transformer-for-affective-behavior-analysis-eg3lugi0.pdf

Spatial-temporal Transformer for Affective Behavior Analysis

Facial expressions convey a great deal of information during human emotional interaction. However, due to the potential for various types of in-the-wild interferences, such as occlusions and variant head poses, dynamic facial expression recognition (DFER) has been a desperately complicated task. Previous methods focus on applying more robust models to extract the spatial-temporal features but ignore the impact of the key features of the regions of interest (ROIs). This inhibits further improvement of recognition accuracy. In this paper, we propose a 3D vision Transformer with a deformable module termed 3D-DSwin Transformer to guide our model to capture more discriminative features. The deformable module can gradually shift the deformable points to guide our model to pay more attention to the ROIs. A simple yet effective video augmentation method is proposed to expand the number of training samples and avoid overfitting. Visualizations and extensive experimental results demonstrate that our proposed 3D-DSwin Transformer has the ability to obtain the key feature maps, and outperforms the previous state-of-the-art methods on both the FERV39k and DFEW benchmarks. 

Dynamic Facial Expression Recognition Based on Vision Transformer with Deformable Module

AMTN: Attention-Enhanced Multimodal Temporal Network for Humor Detection

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Rui Wang

Author Tools

Chat about Author

Papers

Spatial-temporal Transformer for Affective Behavior Analysis

Dynamic Facial Expression Recognition Based on Vision Transformer with Deformable Module

AMTN: Attention-Enhanced Multimodal Temporal Network for Humor Detection