Eye tracking has shown great promise in many scientific fields and daily applications, ranging from the early detection of mental health disorders to foveated rendering in virtual reality (VR). These applications all call for a robust system for high-frequency near-eye movement sensing and analysis in high precision, which cannot be guaranteed by the existing eye tracking solutions with CCD/CMOS cameras. To bridge the gap, in this paper, we propose Swift-Eye, an offline precise and robust pupil estimation and tracking framework to support high-frequency near-eye movement analysis, especially when the pupil region is partially occluded. Swift-Eye is built upon the emerging event cameras to capture the high-speed movement of eyes in high temporal resolution. Then, a series of bespoke components are designed to generate high-quality near-eye movement video at a high frame rate over kilohertz and deal with the occlusion over the pupil caused by involuntary eye blinks. According to our extensive evaluations on EV-Eye, a large-scale public dataset for eye tracking using event cameras, Swift-Eye shows high robustness against significant occlusion. It can improve the IoU and F1-score of the pupil estimation by 20 approach, when over 80 extremely high temporal resolution and can support high-frequency eye movement analysis and a number of potential applications, such as mental health diagnosis, behaviour-brain association, etc. The implementation details and source codes can be found at https://github.com/ztysdu/Swift-Eye.

Swift-Eye: Towards Anti-blink Pupil Tracking for Precise and Robust High-Frequency Near-Eye Movement Analysis with Event Cameras.

Camera-based photoplethysmography (cbPPG) is a non-contact technique that measures cardiac-related blood volume alterations in skin surface vessels through the analysis of facial videos. While traditional approaches can estimate heart rate (HR) under different illuminations, their accuracy can be affected by motion artifacts, leading to poor waveform fidelity and hindering further analysis of heart rate variability (HRV); deep learning-based approaches reconstruct high-quality pulse waveform, yet their performance significantly degrades under illumination variations. In this work, we aim to leverage the strength of these two methods and propose a framework that possesses favorable generalization capabilities while maintaining waveform fidelity. For this purpose, we propose the cbPPGGAN, an enhancement framework for cbPPG that enables the flexible incorporation of both unpaired and paired data sources in the training process. Based on the waveforms extracted by traditional approaches, the cbPPGGAN reconstructs high-quality waveforms that enable accurate HR estimation and HRV analysis. In addition, to address the lack of paired training data problems in real-world applications, we propose a cycle consistency loss that guarantees the time-frequency consistency before/after mapping. The method enhances the waveform quality of traditional POS approaches in different illumination tests (BH-rPPG) and cross-datasets (UBFC-rPPG) with mean absolute error (MAE) values of 1.34 bpm and 1.65 bpm, and average beat-to-beat (AVBB) values of 27.46 ms and 45.28 ms, respectively. Experimental results demonstrate that the cbPPGGAN enhances cbPPG signal quality and outperforms the state-of-the-art approaches in HR estimation and HRV analysis. The proposed framework opens a new pathway toward accurate HR estimation in an unconstrained environment.

cbPPGGAN: A Generic Enhancement Framework for Unpaired Pulse Waveforms in Camera-based Photoplethysmography.

This paper uses Mixed Reality (MR) technologies to provide a seamless integration of digital information in physical environments through human-made annotations. Creating digital annotations of physical objects evokes many challenges for performing (simple) tasks such as adding digital notes and connecting them to real-world objects. For that, we have developed an MR system using the Microsoft HoloLens2 to create semantic representations of contextually-aware real-world objects while interacting with holographic virtual objects. User interaction is enhanced with use of fingers as placeholders for menu items. We demonstrate our approach through two real-world scenarios. We also discuss the challenges for using MR technologies.

HandyNotes: using the hands to create semantic representations of contextually aware real-world objects

The increasing use of high-resolution displays and the demand for interactive frame rates presents a major challenge to widespread adoption of virtual reality. Foveated rendering address this issue by lowering pixel sampling rate at the periphery of the display. How-ever, existing techniques do not fully exploit the feature of human binocular vision, i.e., the dominant eye. In this paper, we propose a Dominant-Eye-Aware foveated rendering method optimized with Multi-Parameter foveation (DEAMP). Specifically, we control the level of foveation for both eyes with two distinct sets of foveation parameters. To achieve this, each eye’s visual field is divided into three nested layers based on eccentricity. Multiple parameters govern the level of foveation of each layer, respectively. We conduct user studies to evaluate our method. Experimental results demonstrate that DEAMP is superior in terms of rendering time and reduces the disparity between pixel sampling rate and the visual acuity fall-off model while maintaining the perceptual quality.

DEAMP: Dominant-Eye-Aware Foveated Rendering with Multi-Parameter optimization

This paper investigates the influence of X-ray vision metaphors on distance estimation in optical-see-through augmented reality (AR) in action space. A within-subjects study $(\mathrm{N}=30)$ was conducted to evaluate depth judgments across five conditions, including a novel “invisible mesh” technique. Participants performed a series of blind walking tasks that required estimating the depth of AR objects displayed at multiple distance ranges in front or behind a physical occluding surface. Although quantitative results regarding the impact of different X-ray vision metaphors on distance perception were inconclusive, participant feedback revealed a diversity of strategies and preferences. Overall, the findings suggest that no single metaphor was considered universally superior, and multiple X-ray vision metaphors may be suitable for different users and situations. This research contributes to understanding of X-ray vision techniques and informs the design considerations for AR systems aiming to enhance depth perception and user experience.

Invisible Mesh: Effects of X-Ray Vision Metaphors on Depth Perception in Optical-See-Through Augmented Reality

Remote photoplethysmography (rPPG) is a non-contact technology that can estimate heart rate using facial video and holds significant potential for health monitoring. Despite the latest deep learning-based rPPG approaches can predict high-quality rPPG signal under similar scenarios, these methods often suffer from degraded performance when encountering variations in subjects, environments, or illumination conditions in target domains. To address this challenge, we propose an uncertainty-guided self-training approach that leverages model uncertainty and periodic priors to enhance generalization across different domains without requiring labels in the target domain. We iteratively update the model using pseudo-labels generated from its own predictions on unlabelled data in the target domain, with varying confidence levels informed by the model’s uncertainty estimation. To achieve this, we modify a standard Convolutional Neural Network (CNN) into a Bayesian Neural Network (BNN) for uncertainty estimation, which guides the assignment of pseudo-labels with varying confidence levels. By employing the adversarially learned periodic priors of rPPG signals shared across domains as regularization terms, we further stabilize the model adaptation process. We evaluate the proposed method on two public datasets (PURE and UBFC-rPPG) across five cross-domain tasks. Experimental results demonstrate improved performance over the baselines, with gains ranging from 60.5% to 97.2%, outperforming existing methods in generalization performance for rPPG-based heart rate measurement.

Towards Practical Facial Video-based Remote Heart Rate Estimation via Cross Domain rPPG Adaptation

Recent research based on hand-eye coordination has shown that gaze could improve object selection and translation experience under certain scenarios in AR. However, several limitations still exist. Specifically, we investigate whether gaze could help object selection with heavy 3D occlusions and help 3D object translation in the depth dimension. In addition, we also investigate the possibility of reducing the gaze calibration burden before use. Therefore, we develop new methods with proper gaze guidance for 3D interaction in AR, and also an implicit online calibration method. We conduct two user studies to evaluate different interaction methods and the results show that our methods not only improve the effectiveness of occluded objects selection but also alleviate the arm fatigue problem significantly in the depth translation task. We also evaluate the proposed implicit online calibration method and find its accuracy comparable to standard 9 points explicit calibration, which makes a step towards practical use in the real world.

Exploring 3D Interaction with Gaze Guidance in Augmented Reality

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Feng Lu

Author Tools

Chat about Author

Papers

DEAMP: Dominant-Eye-Aware Foveated Rendering with Multi-Parameter optimization

Towards Practical Facial Video-based Remote Heart Rate Estimation via Cross Domain rPPG Adaptation

Exploring 3D Interaction with Gaze Guidance in Augmented Reality