Latest developments in computer hardware, sensor technologies, and artificial intelligence can make virtual reality (VR) and virtual spaces an important part of human everyday life. Eye tracking offers not only a hands-free way of interaction but also the possibility of a deeper understanding of human visual attention and cognitive processes in VR. Despite these possibilities, eye-tracking data also reveal privacy-sensitive attributes of users when it is combined with the information about the presented stimulus. To address these possibilities and potential privacy issues, in this survey, we first cover major works in eye tracking, VR, and privacy areas between the years 2012 and 2022. While eye tracking in the VR part covers the complete pipeline of eye-tracking methodology from pupil detection and gaze estimation to offline use and analyses, as for privacy and security, we focus on eye-based authentication as well as computational methods to preserve the privacy of individuals and their eye-tracking data in VR. Later, taking all into consideration, we draw three main directions for the research community by mainly focusing on privacy challenges. In summary, this survey provides an extensive literature review of the utmost possibilities with eye tracking in VR and the privacy implications of those possibilities.

https://export.arxiv.org/pdf/2305.14080v1.pdf

Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges

Grasp intention recognition plays a crucial role in controlling assistive robots to aid older people and individuals with limited mobility in restoring arm and hand function. Among the various modalities used for intention recognition, the eye-gaze movement has emerged as a promising approach due to its simplicity, intuitiveness, and effectiveness. Existing gaze-based approaches insufficiently integrate gaze data with environmental context and underuse temporal information, leading to inadequate intention recognition performance. The objective of this study is to eliminate the proposed deficiency and establish a gaze-based framework for object detection and its associated intention recognition. A novel gaze-based grasp intention recognition and sequential decision fusion framework (GIRSDF) is proposed. The GIRSDF comprises three main components: gaze attention map generation, the Gaze-YOLO grasp intention recognition model, and sequential decision fusion models (HMM, LSTM, and GRU). To evaluate the performance of GIRSDF, a dataset named Invisible containing data from healthy individuals and hemiplegic patients is established. GIRSDF is validated by trial-based and subject-based experiments on Invisible and outperforms the previous gaze-based grasp intention recognition methods. In terms of running efficiency, the proposed framework can run at a frequency of about 22 Hz, which ensures real-time grasp intention recognition. This study is expected to inspire additional gaze-related grasp intention recognition works.

Gaze and Environmental Context-Guided Deep Neural Network and Sequential Decision Fusion for Grasp Intention Recognition.

Gaze is a crucial element in human-computer interaction and plays an increasingly vital role in promoting the adoption of head-mounted devices (HMDs). Existing gaze tracking methods for HMDs either demand user calibration or face challenges in balancing accuracy and speed, compromising the overall user experience. In this paper, we introduce a novel strategy for real-time, calibration-free gaze tracking using joint head-eye cues on HMDs. Initially, we create a multimodal gaze tracking dataset named HE-Gaze, encompassing synchronized eye images and 6DoF head movement data, addressing a gap in the current data landscape. Statistical analyses unveil the correlation between head movements and gaze positions. Building on these insights, we introduce the hierarchical head-eye coordinated gaze tracking model (HHE-Tracker), which incorporates two lightweight branches to encode input eye images and head sequences efficiently. It combines encoded head velocity and posture features with eye features across various scales to infer gaze position. HHE-Tracker was implemented on a commercial HMD, and its performance was assessed in unconstrained scenarios. The results demonstrate the HHE-Tracker's capability to accurately estimate gaze positions in real-time. In comparison to the state-of-the-art gaze tracking algorithm, HHE-Tracker exhibits commendable accuracy (3.47<inline-formula><tex-math notation="LaTeX">$^{\circ }$</tex-math><alternatives><mml:math><mml:msup><mml:mrow/><mml:mo>∘</mml:mo></mml:msup></mml:math><inline-graphic xlink:href="li-ieq1-3425928.gif"/></alternatives></inline-formula>) and a 40-fold speedup (81 <italic>FPS</italic> on a Snapdragon 845 SoC).

Real-time Gaze Tracking via Head-eye Cues on Head Mounted Devices

Viewport prediction is a critical aspect of virtual reality (VR) video streaming, directly impacting user experience in adaptive streaming. However, most existing algorithms treat users as homogeneous entities and overlook the variations in user behaviors and video content. Additionally, they often struggle with long-term predictions and intense movement. Our research sheds light on the importance of considering user behavior variations and leveraging advanced techniques to optimize robust viewport prediction in VR video streaming. First, we address these limitations by conducting a comprehensive feature analysis on existing datasets to uncover distinctive user behaviors. Building upon these findings, we propose a novel approach that utilizes the power of Bidirectional Long Short-Term Memory (BiLSTM) networks and meta-learning. The BiLSTM architecture effectively captures long-term dependencies, which can strengthen the robustness of viewport prediction especially in longterm prediction and intense movement. Additionally, meta-learning enables personalized adaptation to individual users’ viewing behaviors. Through extensive evaluations on diverse datasets, our algorithm Meta360 demonstrates superior performance in terms of accuracy and robustness compared to state-of-the-art methods.

Meta360: Exploring User-Specific and Robust Viewport Prediction in360-Degree Videos through Bi-Directional LSTM and Meta-Adaptation

This paper presents a near-eye DVS-based end-to-end eye tracking processor for augmented reality/virtual reality (AR/VR) devices, addressing the need for low-power, high frame rate and accurate eye tracking deployment on wearable devices with extended battery life. The processor features a dedicated hardware implementation for energy-efficient event-driven pre-processing and acceleration, and delivers accurate gaze estimation utilizing a customized neural network (NN) with constant-event-count sample partitioning method. It is capable of performing end-to-end eye tracking exclusively with the Dynamic Vision Sensor (DVS), achieving an average accuracy of 0.91° in a 96°×64° FoV. The processor is implemented and simulated using UMC 40-nm LP CMOS technology, with a core area of 1.88 mm2. Operating at a clock frequency of 200 MHz, it achieves a dynamic frame rate of up to 1250 Hz with a power dissipation of 8.09 mW, corresponding to a 20.82 µJ energy per gaze estimation. With the integration of DVS, this processor enables real-time, low-power and accurate eye tracking for an immersive interaction experience on AR/VR devices.

A Near-Eye DVS-Based End-to-End Eye Tracking Processor for AR/VR Applications

High-accuracy, low-latency gaze tracking is becoming one of the indispensable features in augmented reality (AR) head-mounted devices (HMDs). Researchers have proposed different approaches to predict gaze positions from eye images. However, since only the eye modality is focused, these appearance-based algorithms are still struggle to trade off the accuracy and running speed in HMDs. In this paper, we propose a lightweight multi-modal network (HE-Tracker) to regress gaze positions. By fusing head-movement features with eye features, HE-Tracker achieves comparable accuracy (3.655° in all subjects) and $27 \times$ speedup (48 fps in the specialized AR HMD) compared to the state-of-the-art gaze tracking algorithm. We further demonstrate that when applying our head-eye coordination strategy to other baseline models, all these models achieve at least 6.36% performance improvement without a pronounced effect on running speed. Moreover, we construct HE-Gaze, the first multi-modal dataset with eye images and head-movement data for near-eye gaze tracking. This dataset is currently made of 757,360 frames and 15 persons, providing an opportunity to foster research in multi-modal gaze tracking approaches. Our dataset is available at DOWNLOAD LINK 1.

Real-time Gaze Tracking with Head-eye Coordination for Head-mounted Displays

Gait anomaly detection based on video-derived 3D pose estimation.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Lingling Chen

Author Tools

Chat about Author

Papers

Real-time Gaze Tracking with Head-eye Coordination for Head-mounted Displays

Gait anomaly detection based on video-derived 3D pose estimation.