From the past few decades, Human activity recognition (HAR) is one of the vital research areas in computer vision in which much research is ongoing. The researcher’s focus is shifting towards this area due to its vast range of real-life applications to assist in daily living. Therefore, it is necessary to validate its performance on standard benchmark datasets and state-of-the-art systems before applying it in real-life applications. The primary objective of this Systematic Literature Review (SLR) is to collect existing research on video-based human activity recognition, summarize, and analyze the state-of-the-art deep learning architectures regarding various methodologies, challenges, and issues. The top five scientific databases (such as ACM, IEEE, ScienceDirect, SpringerLink, and Taylor & Francis) are accessed to accompany this systematic study by summarizing 70 different research articles on human activity recognition after critical review. Human activity recognition in videos is a challenging problem due to its diverse and complex nature. For accurate video classification, extraction of both spatial and temporal features from video sequences is essential. Therefore, this SLR focuses on reviewing the recent advancements in stratified self-deriving feature-based deep learning architectures. Furthermore, it explores various deep learning techniques available for HAR, challenges researchers to face to build a robust model, and state-of-the-art datasets used for evaluation. This SLR intends to provide a baseline for video-based human activity recognition research while emphasizing several challenges regarding human activity recognition accuracy in video sequences using deep neural architectures.

/pdf/analysis-of-deep-neural-networks-for-human-activity-40kl422ooq.pdf

Analysis of Deep Neural Networks for Human Activity Recognition in Videos—A Systematic Literature Review

In most of the existing work for activity recognition, 3D ConvNets show promising performance for learning spatiotemporal features of videos. However, most methods sample fixed-length frames from the original video, which are cropped to a fixed size and fed into the model for training. In this manner, two problems limit the model performance for recognition. First, the cropped video clips are incomplete or even distorted in appearance, resulting in a large gap between the feature representation and semantics of human activity. Second, the useful features of longer video frame sequences are weakened by the repeated stacking of 3D convolution over deep networks due to the limitations of GPU memory and computing ability. This article proposes a method based on a 3D backbone network for multi scale spatial feature representation, which uses a pyramid pooling layer to allow the input of video frames at different scales, and then aggregates short-term spatial–temporal features into a long-term video-level representation. Objection detection is used as a component of model testing to explore the improvement of activity recognition considering the large amount of space–time redundancy in real life videos. An experiment is performed on the principal video dataset, UCF101, and the proposed method presents a competitive performance.

/pdf/complete-video-level-representations-for-action-recognition-143653h0ql.pdf

Complete Video-Level Representations for Action Recognition

Human action recognition is a supervised process of labeling an entire video image sequence with action labels in computer vision fields. Different from the recognition of static images, this process also needs to learn the contact information between video frames, such as the timing characteristics that reflect the changes of actions in the video. In existing deep learning methods, due to the size of the convolution kernel, the models use a small number of consecutive frames as input and trains to assign feature vectors to short sequences instead of the entire sequence. Therefore, even if the learned features contain time information, their evolution over time will be completely ignored. In this work, we propose a dual-stream deep fusion framework that can fully utilize the long-term information of a video. We preprocess the video into static frames and optical flow graphs and input them into a three-dimensional convolutional neural network to obtain the spatiotemporal feature stream with time series. Then, the spatiotemporal feature stream is input into a simple recurrent unit network to learn the long-term sequence features of the time dimension. Finally, SoftMax classifier is used for feature classification. We tested our model on the classic action recognition UCF -101 and HMDB-S1 data sets, and our model achieved better performance than existing methods. 

3D Dual-stream Convolutional Neural Networks with Simple Recurrent Unit Network: A New Framework for Action Recognition

Multi-agent systems are suitable for handling complex problems due to their high parallelism and autonomous evolution ability. In this paper, we propose an adaptive clustering multi-agent learning system for intelligent applications with continuously changing requirements. Each agent model changes temporal sequences using the longest common subsequence (LCS) algorithm. Multiple agents collaborate in a multilayer decentralized approach to enhance learning adaptability and achieve self-supervised behavioral clustering. The system is constructed using a “memory-like” method and operates primarily on memory access and comparison, avoiding extensive matrix operations of artificial neural networks while achieving learning and prediction functions. We chose an unsupervised vehicle behavioral clustering scenario for feasibility validation in which the system’s cognitive objective is to cluster and recognize vehicle behaviors. In a low computational environment, the system can complete clustering functions and exhibit continuous learning capabilities when new behavioral changes occur. The proposed approach achieves an accuracy of 97.4% while processing at a speed 1–5 times faster than similar clustering algorithms. The verification results indicate that this system has excellent potential to enhance intelligent sensing front ends.

/pdf/adaptive-modeling-multi-agent-learning-system-for-video-3k9e7xdg.pdf

Adaptive-Modeling Multi-Agent Learning System for Video Behavioral Clustering Recognition

In recent years, action recognition based on two-stream networks has developed rapidly. However, most existing methods describe incomplete and distorted video content due to cropped and warped frame or clip-level feature extraction. This paper proposed an approach based on deep learning that preserves the complete contextual relation of temporal human actions in videos. The proposed architecture follows the two-stream network with a novel 3D Convolutional Network (ConvNets) and pyramid pooling layer, to design an end-to-end behavioral feature learning method. The 3D ConvNets extract video-level, spatial-temporal features from two input streams, the RGB images and the corresponding optical flow. The multi-scale pyramid pooling layer fixed the generated feature maps into a unified size regardless of input video size. The final predictions are resulted from a fused softmax scores of two streams, and subject to the weighting factor of each stream. Our experimental results suggest spatial stream slightly higher than the temporal stream, and the performance of the trained model is conditionally optimized. The proposed method is experimented on two challenging video action datasets UCF101 and HMDB51, in which our method achieves the most advanced performance above 96.1% on UCF101 dataset.

3D Convolutional Two-Stream Network for Action Recognition in Videos

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Min Li

Author Tools

Chat about Author

Papers

3D Convolutional Two-Stream Network for Action Recognition in Videos