Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph
AmirAli Bagher Zadeh,Paul Pu Liang,Soujanya Poria,Erik Cambria,Louis-Philippe Morency +4 more
- 01 Jul 2018
- Vol. 1, pp 2236-2246
TL;DR: This paper introduces CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI), the largest dataset of sentiment analysis and emotion recognition to date and uses a novel multimodal fusion technique called the Dynamic Fusion Graph (DFG), which is highly interpretable and achieves competative performance when compared to the previous state of the art.
read more
Abstract: Analyzing human multimodal language is an emerging area of research in NLP Intrinsically this language is multimodal (heterogeneous), sequential and asynchronous; it consists of the language (words), visual (expressions) and acoustic (paralinguistic) modalities all in the form of asynchronous coordinated sequences From a resource perspective, there is a genuine need for large scale datasets that allow for in-depth studies of this form of language In this paper we introduce CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI), the largest dataset of sentiment analysis and emotion recognition to date Using data from CMU-MOSEI and a novel multimodal fusion technique called the Dynamic Fusion Graph (DFG), we conduct experimentation to exploit how modalities interact with each other in human multimodal language Unlike previously proposed fusion techniques, DFG is highly interpretable and achieves competative performance when compared to the previous state of the art
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
New Benchmark Dataset and Fine-Grained Cross-Modal Fusion Framework for Vietnamese Multimodal Aspect-Category Sentiment Analysis
Quy T. Nguyen,Manh Duc Nguyen,Kiet Van Nguyen +2 more
- 01 May 2024
TL;DR: New benchmark dataset and fine-grained cross-modal fusion framework for Vietnamese multimodal aspect-category sentiment analysis. The dataset includes 4,876 text-image pairs with fine-grained annotations for text and image. The framework effectively learns intra- and inter-modality interactions and achieves the highest F1 score of 79.73%.
Modality-specific and -shared Contrastive Learning for Sentiment Analysis
Jiuxiang You,Guobo Xie,Fu Lee Wang,Zhenguo Yang +3 more
- 30 May 2024
TL;DR: MMCL achieves state-of-the-art performance for sentiment analysis using modality-specific and -shared contrastive learning.
Sentiment Analysis: Comprehensive Reviews, Recent Advances, and Open Challenges.
Qiang Lu,Xia Sun,Yunfei Long +2 more
TL;DR: In this paper , a taxonomy of multimodal Sentiment Analysis (MSA) methods is proposed to provide a comprehensive understanding of relevant advances in the field of sentiment analysis.
Multimodal Emotion Recognition Dataset in the Wild (MERDWild)
Facundo Martínez,Ana Aguilera,Diego Mellado +2 more
- 05 Dec 2023
TL;DR: MERDWild, a multimodal emotion recognition dataset, addresses the challenge of unifying, cleaning, and transforming three datasets collected in uncontrolled environments with the aim of integrating and standardizing a database that encompasses three modalities: facial images, audio, and text.
Machine Learning Approaches for Applied Affective Computing
25 Jan 2022
TL;DR: Tian et al. as discussed by the authors proposed Machine Learning Approaches for Applied Affective Computing Authors: Leimin Tian Monash University Monash university search about this author , Sharon Oviatt Monash United States Naval Academy (US Naval Academy), Monash, Australia search about the author , Michal Muszynski Carnegie Mellon University and University of Geneva Carnegie Mellon and University GenevaSearch about this article , Brent C. Chamberlain Utah State University Utah State universitySearch about that author , Jennifer Healey Adobe Research, San Jose Adobe Research.
References
Random Forests
Leo Breiman
- 01 Oct 2001
TL;DR: Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the forest, and are also applicable to regression.
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
99K
Support-Vector Networks
Corinna Cortes,Vladimir Vapnik +1 more
TL;DR: High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated and the performance of the support- vector network is compared to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
Glove: Global Vectors for Word Representation
Jeffrey Pennington,Richard Socher,Christopher D. Manning +2 more
- 01 Oct 2014
TL;DR: A new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods and produces a vector space with meaningful substructure.
•Posted Content
TensorFlow: A system for large-scale machine learning
Martín Abadi,Paul Barham,Jianmin Chen,Zhifeng Chen,Andy Davis,Jeffrey Dean,Matthieu Devin,Sanjay Ghemawat,Geoffrey Irving,Michael Isard,Manjunath Kudlur,Josh Levenberg,Rajat Monga,Sherry Moore,Derek G. Murray,Benoit Steiner,Paul A. Tucker,Vijay K. Vasudevan,Pete Warden,Martin Wicke,Yuan Yu,Xiaoqiang Zheng +21 more
TL;DR: The TensorFlow dataflow model is described and the compelling performance that Tensor Flow achieves for several real-world applications is demonstrated.