Gaze360: Physically Unconstrained Gaze Estimation in the Wild
Petr Kellnhofer,Adrià Recasens,Simon Stent,Wojciech Matusik,Antonio Torralba +4 more
- 01 Oct 2019
- pp 6912-6921
TL;DR: Gaze360 as discussed by the authors is a large-scale remote gaze tracking dataset and method for robust 3D gaze estimation in unconstrained images, which consists of 238 subjects in indoor and outdoor environments with labelled three-dimensional (3D) gaze across a wide range of head poses and distances.
read more
Abstract: Understanding where people are looking is an informative social cue. In this work, we present Gaze360, a large-scale remote gaze-tracking dataset and method for robust 3D gaze estimation in unconstrained images. Our dataset consists of 238 subjects in indoor and outdoor environments with labelled 3D gaze across a wide range of head poses and distances. It is the largest publicly available dataset of its kind by both subject and variety, made possible by a simple and efficient collection method. Our proposed 3D gaze model extends existing models to include temporal information and to directly output an estimate of gaze uncertainty. We demonstrate the benefits of our model via an ablation study, and show its generalization performance via a cross-dataset evaluation against other recent gaze benchmark datasets. We furthermore propose a simple self-supervised approach to improve cross-dataset domain adaptation. Finally, we demonstrate an application of our model for estimating customer attention in a supermarket setting. Our dataset and models will be made available at http://gaze360.csail.mit.edu.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
AV-GAZE: A Study on the Effectiveness of Audio Guided Visual Attention Estimation for Non-profilic Faces
16 Oct 2022
TL;DR: In this paper , the authors explore if audio-guided coarse head-pose can further enhance visual attention estimation performance for non-prolific faces by using off-the-shelf state-of-theart models to facilitate cross-modal weak-supervision.
PCFGaze: Physics-Consistent Feature for Appearance-based Gaze Estimation
Yiwei Bao,Feng Lu +1 more
TL;DR: This paper analyzes the gaze feature manifold to construct the Physics- Consistent Feature (PCF) in an analytical way, which connects gaze feature to the physical definition of gaze, and proposes the PCFGaze framework that directly optimizes gaze feature space by the guidance of PCF.
RavenGaze: A Dataset for Gaze Estimation Leveraging Psychological Experiment Through Eye Tracker
05 Jan 2023
TL;DR: Zhang et al. as discussed by the authors designed an experiment employing Raven's Matrices as visual stimuli and collecting gaze data, facial videos as well as screen content videos simultaneously, and the results show that the existing algorithms perform well on their RavenGaze dataset in the 3D and 2D gaze estimation task, and demonstrate good generalization ability according to cross-dataset evaluation task.
1
Gaze Target Detection by Merging Human Attention and Activity Cues
Yaokun Yang,Yihan Yin,Feng Lu +2 more
- 24 Mar 2024
TL;DR: This study introduces a novel approach that merges visual saliency detection with human attention and activity cues, achieving state-of-the-art performance on gaze target detection tasks, outperforming 3D reconstruction-based methods and demonstrating human-level performance with 2D image information.
References
Deep Residual Learning for Image Recognition
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
•Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
- 01 Jan 2015
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
138.5K
•Posted Content
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: In this article, the adaptive estimates of lower-order moments are used for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimate of lowerorder moments.
82.5K
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Zhe Cao,Tomas Simon,Shih-En Wei,Yaser Sheikh +3 more
- 21 Jul 2017
TL;DR: Part Affinity Fields (PAFs) as discussed by the authors uses a nonparametric representation to learn to associate body parts with individuals in the image and achieves state-of-the-art performance on the MPII Multi-Person benchmark.
6.2K
Adversarial Discriminative Domain Adaptation
Eric Tzeng,Judy Hoffman,Kate Saenko,Trevor Darrell +3 more
- 21 Jul 2017
TL;DR: Adversarial Discriminative Domain Adaptation (ADDA) as mentioned in this paper combines discriminative modeling, untied weight sharing, and a generative adversarial network (GAN) loss.