With the rapid development in social media, single-modal emotion recognition is hard to satisfy the demands of the current emotional recognition system. Aiming to optimize the performance of the emotional recognition system, a multimodal emotion recognition model from speech and text was proposed in this paper. Considering the complementarity between different modes, CNN (convolutional neural network) and LSTM (long short-term memory) were combined in a form of binary channels to learn acoustic emotion features; meanwhile, an effective Bi-LSTM (bidirectional long short-term memory) network was resorted to capture the textual features. Furthermore, we applied a deep neural network to learn and classify the fusion features. The final emotional state was determined by the output of both speech and text emotion analysis. Finally, the multimodal fusion experiments were carried out to validate the proposed model on the IEMOCAP database. In comparison with the single modal, the overall recognition accuracy of text increased 6.70%, and that of speech emotion recognition soared 13.85%. Experimental results show that the recognition accuracy of our multimodal is higher than that of the single modal and outperforms other published multimodal models on the test datasets.

/pdf/audio-textual-emotion-recognition-based-on-improved-neural-1rdubuhwek.pdf

Audio-Textual Emotion Recognition Based on Improved Neural Networks

In recent years, skeleton-based human action recognition has attracted substantial attentions. However, owing to the complexity and nonlinearity of human action data, it is still a challenging task to precisely represent skeleton features. Motivated by the effectiveness of Lie Group skeletal representations in extracting human action features and the powerful capability of deep neural networks in feature learning and high-dimensional data processing, we proposed to combine Lie Group features and deep learning for human action recognition. Human skeleton information was firstly used to overcome the interference of external factors such as changes of lighting conditions and body shape. And then, Lie Group was applied to naturally represent the complex and diverse action data. Finally, we took use of convolutional neural networks to learn and classify the Lie Group features. Experiments were performed on three public datasets, and the experimental results show that our methods can achieve higher average recognition accuracy of 93.00% on Florence3D-Action, 93.68% on MSR Action Pairs, and 97.96% on UT Kinect-Action, which outperforms many of the state-of-the-art methods.

Human action recognition using Lie Group features and convolutional neural networks

Cross-view action recognition with small-scale datasets

The emergence of low-cost depth cameras creates potential for RGB-D based human action recognition. However, most of the existing RGB-D based approaches simply concatenate original heterogeneous features without discovering the latent relations among different modalities. We propose a discriminative common structure learning (DCSL) model for human action recognition from RGB-D sequences. Specifically, we extract deep learning-based features and hand-crafted features from multimodal data (skeleton, depth, and RGB). In particular, we propose a deep architecture based on 3-D convolutional neural network to automatically extract deep spatiotemporal features from raw sequences. The proposed DCSL model utilizes a generalized version of collective matrix factorization to learn shared features among different modalities. To perform supervised learning and preserve intermodal similarity, we formulate a graph regularization term by considering both label information and similar geometric structure of multimodal data, which intends to improve the discriminative power of shared features. Moreover, we solve the objective function using an iterative optimization algorithm. Then, an improved collaborative representation classifier is employed to perform computationally efficient action recognition. Experimental results on four action datasets demonstrate the superior performance of the proposed method.

RGB-D action recognition based on discriminative common structure learning model

Ever advancing development in Computer Vision and Deep Learning has increased the efficacy of smart monitoring by analyzing and predicting the physical abnormalities and generating time-sensitive results. Based on the improved principles of smart monitoring and data processing, a novel computer vision assisted deep learning based posture monitoring system is proposed to predict Generalized Anxiety Disorder (GAD) oriented physical abnormalities of an individual from their working environment. We used deep learning-assisted 3D Convolutional Neural Network (CNN) technology for spatio-temporal feature extraction and Gated Recurrent Unit (GRU) model to exploit the extracted temporal dynamics for adversity scale determination. The alert-based decisions with the deliverance of the physical state helps to increase the utility of the proposed system in the healthcare or assistive-care domain. The proficiency of the system is also enhanced by storing the predicted anomaly scores in the local database of the system which can be further used for therapeutic purposes. To validate the prediction performance of the proposed system, extensive experiments are conducted on three challenging datasets, NTU RGB+D, UTD-MHAD and HMDB51. The proposed methodology achieved comparable performance by obtaining the mean accuracy of 91.88%, 94.28%, and 70.33%, respectively. Furthermore, the average prediction time taken by the proposed methodology is approximately 1.13 seconds which demonstrates the real-time monitoring efficiency of the system. The calculated outcomes show that the proposed methodology performs better contrasted with other contemporary studies for activity prediction, data processing cost, error rate, and time complexity.

Computer vision based working environment monitoring to analyze Generalized Anxiety Disorder (GAD)

Human action recognition has been widely used in various fields of computer vision, pattern recognition, and human–computer interaction and has attracted substantial attention. Combining deep learning and depth information, this paper proposed a method of human action recognition based on improved convolutional neural networks (CNN). First, we use the depth motion maps to extract the depth sequence features and obtain three projected maps corresponding to front, side, and the top views. On this basis, an improved CNN is constructed to realize the recognition of human action, which uses three-dimensional (3-D) input and two-dimensional process identification to speed up the computation and reduce the complexity of recognition process. We evaluate our approach on two public 3-D action datasets: MSR Action3D dataset and UT-Kinect dataset, and our private CTP Action3D dataset built using Kinect to collect data. The experimental results show that the proposed methods of human action recognition achieve higher average recognition rate of 91.3% on MSR Action3D dataset, 97.98% on UT-Kinect dataset, and the average recognition rate is 93.8% on our CTP Action3D dataset. Furthermore, the trained model on one depth video sequence dataset can be easily generalized to different datasets without changing network parameters.

Robust human action recognition based on depth motion maps and improved convolutional neural network

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Min Xiang

Author Tools

Chat about Author

Papers

Robust human action recognition based on depth motion maps and improved convolutional neural network