Pedestrian trajectory prediction in crowd scenes plays a significant role in intelligent transportation systems. The main challenges are manifested in learning motion patterns and addressing future uncertainty. Typically, trajectory prediction is considered in two dimensions, including temporal dynamics modeling and social interactions capturing. For temporal dependencies, although existing models based on recurrent neural networks (RNNs) or convolutional neural networks (CNNs) achieve high performance on short-term prediction, they still suffer from limited scalability for long sequences. For social interactions, previous graph-based methods only consider fixed features but ignore dynamic interactions between pedestrians. Considering that the transformer network has a strong capability of capturing spatial and long-term temporal dynamics, we propose Long-Short Term Spatio-Temporal Aggregation (LSSTA) network for human trajectory prediction. First, a modern variant of graph neural networks, named spatial encoder, is presented to characterize spatial interactions between pedestrians. Second, LSSTA utilizes a transformer network to handle long-term temporal dependencies and aggregates the spatial and temporal features with a temporal convolution network (TCN). Thus, TCN is combined with the transformer to form a long-short term temporal dependency encoder. Additionally, multi-modal prediction is an efficient way to address future uncertainty. Existing auto-encoder modules are extended with static scene information and future ground truth for multi-modal trajectory prediction. Experimental results on complex scenes demonstrate the superior performance of our method in comparison to existing approaches. 

Long-Short Term Spatio-Temporal Aggregation for Trajectory Prediction

Skeleton-based action recognition has attracted more and more attention due to its advantage in avoiding environmental noise including light transformations and viewpoint changing. Most current approaches consist of interleaving spatial-only modules and temporal-only modules, where messages are first spatially aggregated between human joints and then temporally delivered. Although efficient, such factorized design hinders direct information flow among joints in adjacent frames, thus failing to (1) effectively convey information in a larger neighborhood and (2) be inferior to capture short-term motion to distinguish similar action pairs. To handle these limitations, we propose spatial–temporal graph attention network (STGAT) to dynamically model local cross-spacetime information. Keeping the temporal-only modules unchanged, it inflates the spatial-only modules to perform spatial–temporal modeling from the local spatial–temporal neighborhood without a second transmission cost. Especially, different from previous spatial–temporal modeling methods, graph edges are adaptively computed to dynamically aggregate beneficial information to capture short-term movements, getting rid of predefined fixed connectivity. This ensures STGAT could dynamically attend to important joints in a local spatial–temporal neighborhood to capture critical short-term movements. While STGAT is theoretically effective in introducing local spatial–temporal information by comprising a large spatial–temporal neighborhood, we find its effectiveness is impeded by the inherent redundancy in local features. We propose three simple modules to reduce local feature redundancy to calibrate the feature modeling schema and further release the potential of STGAT, which (1) narrow the scope of self-attention operators, (2) dynamically weight joints along the temporal dimension, and (3) separate subtle motion from static features, respectively. As a result, STGAT achieves state-of-the-art performance on three large-scale datasets: NTU RGB＋D 60, NTU RGB＋D 120, and Kinetics Skeleton 400. Thanks to special care on local cross-spacetime information, STGAT generalizes better upon classifying hard similar actions than previous methods. Visualizations demonstrate that STGAT could intelligently attend to critical joints when dealing with different actions. 

Skeleton-based action recognition with local dynamic spatial–temporal aggregation

Worsening vision at age 4–5 in England post‐COVID: Evidence from a large database of vision screening data

Currently, video human behaviour recognition is the most foundational task of computer vision. The conventional recognition frameworks are building based on the images only, with current the wide usage of surveillance video as well as human behaviours are increasingly related to temporal information, video-based behaviour recognition has been widely studied by current researchers. This paper addresses multiple current research works and does the comparison between the variants of these algorithms. The methodologies are considered to be separated into traditional recognition and deep learning based methods which have reached the highest accuracy during current research works. Our paper does the comparison of existing frameworks and datasets which are related to video-type datasets only. We investigated multiple types of neural networks which are utilized for behaviour information extraction, as well as the challenges facing for our conventional and current methods. The deep learning based methodology for extracting both spatialtemporal information has involved plenty of frameworks. We compared the pros and cons of current existing works and provide the further research directions based on existing works 

A survey of video human behaviour recognition Methodologies in the Perspective of Spatial-Temporal

A unified framework for unsupervised action learning via global-to-local motion transformer

Graph convolutional networks (GCNs) achieve promising performance for skeleton-based action recognition. However, in most GCN-based methods, the spatial-temporal graph convolution is strictly restricted by the graph topology while only captures the short-term temporal context, thus lacking the flexibility of feature extraction. In this work, we present a novel architecture, named Graph Convolutional skeleton Transformer (GCsT), which addresses limitations in GCNs by introducing Transformer. Our GCsT employs all the benefits of Transformer (i.e. dynamical attention and global context) while keeps the advantages of GCNs (i.e. hierarchy and local topology structure). In GCsT, the spatial-temporal GCN forces the capture of local dependencies while Transformer dynamically extracts global spatial-temporal relationships. Furthermore, the proposed GCsT shows stronger expressive capability by adding additional information present in skeleton sequences. Incorporating the Transformer allows that information to be introduced into the model almost effortlessly. We validate the proposed GCsT by conducting extensive experiments, which achieves the state-of-the-art performance on NTU RGB+D, NTU RGB+D 120 and Northwestern-UCLA datasets.

GCsT: Graph Convolutional Skeleton Transformer for Action Recognition.

Vision Transformers (ViTs) are promising for solving video-related tasks, but often suffer from computational bottlenecks or insufficient temporal information. Recent advances in large-scale pre-training show great potential for high-quality video representation, providing new remedies to transformer limitations. Inspired by this, we propose a SpatioTemporal Representation Enhanced Vision Transformer (STRE-ViT), which follows a two-stream paradigm to fuse large-scale pre-training visual prior knowledge and video-level temporal biases in a simple and effective manner. Specifically, one stream employs a well-pretrained ViT with rich vision priors to alleviate data requirements and learning workload. Another stream is our designed spatiotemporal interaction stream, which first models video-level temporal dynamics and then extracts fine-grained and salient spatiotemporal representations by introducing appropriate temporal bias. Through this interaction stream, the model capacity of ViT is enhanced for video spatiotemporal representations. Moreover, we provide a fresh perspective to adapt well-pretrained ViT for video recognition. Experimental results show that STRE-ViT learns high-quality video representations and achieves competitive performance on two popular video benchmarks. 

Spatiotemporal Representation Enhanced ViT for Video Recognition

In recent years, vision screening has emerged globally for employment (on a yearly basis) within primary and high schools since myopia heavily affects school-aged children. However, this is a laborious and time-consuming task. This article proposes an intelligent system for "self-service" vision screening. Individuals can accomplish this task independently-without any assistance by technical staff. The technical solution involved within this platform is human action recognition realized by pose estimation (real-time human joint localization in images, including detection, association, and tracking). The developed system is based on a compact and embedded artificial intelligence platform, aided by a red-green-blue-D sensor for ranging and pose extraction. A set of intuitive upper-limb actions was designed for unambiguous recognition and interaction. The deployment of this intelligent system brings great convenience for large-scale and rapid vision screening. Implementation details were extensively described, and the experimental results demonstrated efficiency for the proposed technique.

Embedded AI system for interactive vision screen based on human action recognition.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Fengfa Li

Author Tools

Chat about Author

Papers

GCsT: Graph Convolutional Skeleton Transformer for Action Recognition.

Spatiotemporal Representation Enhanced ViT for Video Recognition

Embedded AI system for interactive vision screen based on human action recognition.