Proceedings Article10.1145/3503161.3551602
Rethinking Optical Flow Methods for Micro-Expression Spotting
Yuan Zhao,Xin Tong,Zichong Zhu,Jianda Sheng,Lei Dai,Lingling Xu,Xuehai Xia,Jiao Li +7 more
- 10 Oct 2022
18
TL;DR: This paper refines every step of the workflow before feature extraction, which can reduce error propagation and takes the advantage of high-quality alignment method, more accurate landmark detector, and also more robust optical flow estimation.
read more
Abstract: Micro-expressions (MEs) spotting is popular in some fields, for example, criminal investigation and business communication. But it is still a challenging task to spot the onset and offset of MEs accurately in long videos. This paper refines every step of the workflow before feature extraction, which can reduce error propagation. The workflow takes the advantage of high-quality alignment method, more accurate landmark detector, and also more robust optical flow estimation. Besides, Bayesian optimization hybrid with Nash equilibrium is constructed to search for the optimal parameters. It uses two players to optimize two types of parameters, one player is used to control the ME peak spotting, and another for optical flow field extraction. The algorithm can reduce the search space for each player with better generalization. Finally, our spotting method is evaluated on MEGC2022 spotting task, and achieves F1-score 0.3564 on CAS(ME)3-UNSEEN and F1-score 0.3265 on SAMM-UNSEEN.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
SFAMNet: A scene flow attention-based micro-expression network
Gen-Bing Liong,Sze-Teng Liong,Chee Seng Chan,John See +3 more
TL;DR: This paper proposes SFAMNet, an attention-based network that integrates scene flow and RGB-D data to spot and recognize micro-expressions, achieving state-of-the-art performance on ME spotting, recognition, and analysis tasks with a novel data augmentation strategy.
6
SL-Swin: A Transformer-Based Deep Learning Approach for Macro- and Micro-Expression Spotting on Small-Size Expression Datasets
Qianru Chen,Qinghua Zhong +1 more
TL;DR: SL-Swin this paper uses the Swin Transformer network to predict the probability of a frame being within an expression interval, which can be used in a broad range of potential applications such as lie detection and policing.
Efficient Micro-Expression Spotting Based on Main Directional Mean Optical Flow Feature
Jun Yu,Zhongpeng Cai,Shenshen Du,Xiaxin Shen,Lei Wang,Fang Gao +5 more
- 26 Oct 2023
TL;DR: This paper proposes an efficient framework for the expression spotting task that consists of three main modules: Face Cropping and Alignment Module (FCAM), optical flow Feature Extraction Module (FEM), and expression Proposal Generation Module (PGM).
5
FESNet: Spotting Facial Expressions Using Local Spatial Discrepancy and Multi-Scale Temporal Aggregation
Bohao Zhang,Jiale Lu,Changbo Wang,Gaoqi He +3 more
TL;DR: FESNet efficiently spots facial expressions by modeling subtle facial motion as local spatial discrepancy and incorporating temporal correlation through multi-scale temporal aggregation.
SL-Swin: A Transformer-Based Deep Learning Approach for Macro- and Micro-Expression Spotting on Small-Size Expression Datasets
01 Jun 2023
TL;DR: Wang et al. as mentioned in this paper proposed a Transformer-based deep learning approach that predicts a score indicating the probability of a frame being within an expression interval, which achieved an overall F1-score of 0.1366.
2
References
•Proceedings Article
Practical Bayesian Optimization of Machine Learning Algorithms
Jasper Snoek,Hugo Larochelle,Ryan P. Adams +2 more
- 03 Dec 2012
TL;DR: This work describes new algorithms that take into account the variable cost of learning algorithm experiments and that can leverage the presence of multiple cores for parallel experimentation and shows that these proposed algorithms improve on previous automatic procedures and can reach or surpass human expert-level optimization for many algorithms.
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Eddy Ilg,Nikolaus Mayer,Tonmoy Saikia,Margret Keuper,Alexey Dosovitskiy,Thomas Brox +5 more
- 21 Jul 2017
TL;DR: The concept of end-to-end learning of optical flow is advanced and it work really well, and faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet are presented.
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
Deqing Sun,Xiaodong Yang,Ming-Yu Liu,Jan Kautz +3 more
- 01 Jun 2018
TL;DR: PWC-Net as discussed by the authors uses the current optical flow estimate to warp the CNN features of the second image, which is processed by a CNN to estimate the optical flow, and achieves state-of-the-art performance on the MPI Sintel final pass and KITTI 2015 benchmarks.
Two-frame motion estimation based on polynomial expansion
Gunnar Farnebäck
- 29 Jun 2003
TL;DR: A method to estimate displacement fields from the polynomial expansion coefficients is derived and after a series of refinements leads to a robust algorithm that shows good results on the Yosemite sequence.
•Posted Content
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
TL;DR: FlowNet 2.0 as discussed by the authors proposes an end-to-end learning framework for optical flow estimation, which is only marginally slower than the original FlowNet but decreases the estimation error by more than 50%.
1.9K