Artificial intelligence’s rapid advancement has enabled various applications, including intelligent video surveillance systems, assisted living, and human–computer interaction. These applications often require one core task: video-based human action recognition. Research in human video-based human action recognition is vast and ongoing, making it difficult to assess the full scope of available methods and current trends. This survey concisely explores the vision-based human action recognition field and defines core concepts, including definitions and explanations of the common challenges and most used datasets. Additionally, we provide in an easy-to-understand manner the literature approaches and their evolution over time, emphasizing intuitive notions. Finally, we explore current research directions and potential future paths. The core goal of this work is to provide future works with a shared understanding of fundamental ideas and clear intuitions about current works and find new research opportunities.

/pdf/an-overview-of-the-vision-based-human-action-recognition-2ji75jd2.pdf

An Overview of the Vision-Based Human Action Recognition Field

Thermal comfort optimization is key to ensuring the well-being of building occupants and promoting intelligent energy use in the built environment. In the current space of thermal comfort analysis, many techniques are used ranging from more intrusive wearable sensors and qualitative occupant surveys to less intrusive infrared thermal monitoring and human action recognition (HAR). However, as these methods increase surveillance of building occupants in often complex environments, privacy preservation and accurate analysis are essential for optimal thermal control. This paper focuses on uplifting the ability of skeleton-based HAR for use in thermal comfort-related action recognition as this method has been shown to have promising action recognition accuracy while maintaining user privacy. While we focus here on thermal comfort, our analysis and emphasis on the use and advancement of privacy-preserving technologies naturally extend to other systems (e.g., cyber-physical-social infrastructure systems) underpinned by human-centered, or “social,” objectives and interactions. We benchmark several fundamental deep learning models in the skeleton-based HAR space and compare their performance on a new dataset of thermal comfort-related actions.

Skeleton-based Human Action Recognition in a Thermal Comfort Context

The assessment of rehabilitation exercises for neurological and musculoskeletal disorders are crucial for recovery. Traditionally, assessment methods have been subjective, with inherent uncertainty and limitations. This paper introduces a novel multi-modality dataset named FineRehab§ to prompt the study of rehabilitation movement analysis, leveraging advancements in sensor technology and artificial intelligence. FineRehab collects 16 actions from 50 participants, including both patients with musculoskeletal disorders and healthy individuals, and consists of 4,215 action samples captured by two Kinect cameras and 17 IMUs. To benchmark FineRehab, we present a reliable approach to analyze rehabilitation exercises, and make experiments to evaluate the comprehensive movement quality from across multi-dimensions. Comparative experimental analyses have verified the validity of our dataset in distinguishing between the movement of the normal population and patients, which can offer a quantifiable basis for personalized rehabilitation feedback. The introduction of FineRehab will encourage researchers to apply, develop and adapt various methods for rehabilitation exercise analysis.

FineRehab: A Multi-modality and Multi-task Dataset for Rehabilitation Analysis

Gait recognition holds the promise to robustly identify subjects based on walking patterns instead of appearance information. In recent years, this field has been dominated by learning methods based on two principal input representations: dense silhouette masks or sparse pose keypoints. In this work, we propose a novel, point-based Contour-Pose representation, which compactly expresses both body shape and body parts information. We further propose a local-to-global architecture, called GaitContour, to leverage this novel representation and efficiently compute subject embedding in two stages. The first stage consists of a local transformer that extracts features from five different body regions. The second stage then aggregates the regional features to estimate a global human gait representation. Such a design significantly reduces the complexity of the attention operation and improves efficiency and performance simultaneously. Through large scale experiments, GaitContour is shown to perform significantly better than previous point-based methods, while also being significantly more efficient than silhouette-based methods. On challenging datasets with significant distractors, GaitContour can even outperform silhouette-based methods.

pdf/gaitcontour-efficient-gait-recognition-based-on-a-contour-czmsvj3zlo.pdf

GaitContour: Efficient Gait Recognition based on a Contour-Pose Representation

Gait recognition aims to identify a person based on their walking sequences, serving as a useful biometric modality as it can be observed from long distances without requiring cooperation from the subject. In representing a person's walking sequence, silhouettes and skeletons are the two primary modalities used. Silhouette sequences lack detailed part information when overlapping occurs between different body segments and are affected by carried objects and clothing. Skeletons, comprising joints and bones connecting the joints, provide more accurate part information for different segments; however, they are sensitive to occlusions and low-quality images, causing inconsistencies in frame-wise results within a sequence. In this paper, we explore the use of a two-stream representation of skeletons for gait recognition, alongside silhouettes. By fusing the combined data of silhouettes and skeletons, we refine the two-stream skeletons, joints, and bones through self-correction in graph convolution, along with cross-modal correction with temporal consistency from silhouettes. We demonstrate that with refined skeletons, the performance of the gait recognition model can achieve further improvement on public gait recognition datasets compared with state-of-the-art methods without extra annotations.

GaitSTR: Gait Recognition With Sequential Two-Stream Refinement

Most existing gait recognition methods are appearance-based, which rely on the silhouettes extracted from the video data of human walking activities. The less-investigated skeleton-based gait recognition methods directly learn the gait dynamics from 2D/3D human skeleton sequences, which are theoretically more robust solutions in the presence of appearance changes caused by clothes, hairstyles, and carrying objects. However, the performance of skeleton-based solutions is still largely behind the appearance-based ones. This paper aims to close such performance gap by proposing a novel network model, GaitMixer, to learn more discriminative gait representation from skeleton sequence data. In particular, GaitMixer follows a heterogeneous multi-axial mixer architecture, which exploits the spatial self-attention mixer followed by the temporal large-kernel convolution mixer to learn rich multi-frequency signals in the gait feature maps. Experiments on the widely used gait database, CASIA-B, demonstrate that GaitMixer outperforms the previous SOTA skeleton-based methods by a large margin while achieving a competitive performance compared with the representative appearance-based solutions. Code will be available at https://github.com/exitudio/gaitmixer

GaitMixer: Skeleton-based Gait Representation Learning via Wide-spectrum Multi-axial Mixer

Recently, there has been a remarkable increase in the interest towards skeleton-based action recognition within the research community, owing to its various advantageous features, including computational efficiency, representative features, and illumination invariance. Despite this, researchers continue to explore and investigate the most optimal way to represent human actions through skeleton representation and the extracted features. As a result, the growth and availability of human action recognition datasets have risen substantially. In addition, deep learning-based algorithms have gained widespread popularity due to the remarkable advancements in various computer vision tasks. Most state-of-the-art contributions in skeleton-based action recognition incorporate a Graph Neural Network (GCN) architecture for representing the human body and extracting features. Our research demonstrates that Convolutional Neural Networks (CNNs) can attain comparable results to GCN, provided that the proper training techniques, augmentations, and optimizers are applied. Our approach has been rigorously validated, and we have achieved a score of 95% on the NTU-60 dataset

Skeleton-based Human Action Recognition via Convolutional Neural Networks (CNN)

mmWave radar-based gait recognition is a novel user identification method that captures human gait biometrics from mmWave radar return signals. This technology offers privacy protection and is resilient to weather and lighting conditions. However, its generalization performance is yet unknown and limits its practical deployment. To address this problem, in this paper, a non-synthetic dataset is collected and analyzed to reveal the presence of spatial and temporal domain shifts in mmWave gait biometric data, which significantly impacts identification accuracy. To mitigate this issue, a novel self-aligned domain adaptation method called GaitSADA is proposed. GaitSADA improves system generalization performance by using a two-stage semi-supervised model training approach. The first stage employs semi-supervised contrastive learning to learn a compact gait representation from both source and target domain data, aligning source-target domain distributions implicitly. The second stage uses semi-supervised consistency training with centroid alignment to further close source-target domain gap by pseudo-labelling the target-domain samples, clustering together the samples belonging to the same class but from different domains, and pushing the class centroid close to the weight vector of each class. Experiments show that GaitSADA outperforms representative domain adaptation methods with an improvement ranging from 15.41\% to 26.32\% on average accuracy in low data regimes. Code and dataset will be available at https://exitudio.github.io/GaitSADA

GaitSADA: Self-Aligned Domain Adaptation for mmWave Gait Recognition

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ayman Ali

Author Tools

Chat about Author

Papers

GaitMixer: Skeleton-based Gait Representation Learning via Wide-spectrum Multi-axial Mixer

Skeleton-based Human Action Recognition via Convolutional Neural Networks (CNN)

GaitSADA: Self-Aligned Domain Adaptation for mmWave Gait Recognition