Brain-machine interfaces (BMIs) combine methods, approaches, and concepts derived from neurophysiology, computer science, and engineering in an effort to establish real-time bidirectional links bet...

Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation

Appearance-based gaze estimation has attracted more and more attention because of its wide range of applications. The use of deep convolutional neural networks has improved the accuracy significantly. In order to improve the estimation accuracy further, we focus on extracting better features from eye images. Relatively large changes in gaze angles may result in relatively small changes in eye appearance. We argue that current architectures for gaze estimation may not be able to capture such small changes, as they apply multiple pooling layers or other downsampling layers so that the spatial resolution of the high-level layers is reduced significantly. To evaluate whether the use of features extracted at high resolution can benefit gaze estimation, we adopt dilated-convolutions to extract high-level features without reducing spatial resolution. In cross-subject experiments on the Columbia Gaze dataset for eye contact detection and the MPIIGaze dataset for 3D gaze vector regression, the resulting Dilated-Nets achieve significant (up to 20.8%) gains when compared to similar networks without dilated-convolutions. Our proposed Dilated-Net achieves state-of-the-art results on both the Columbia Gaze and the MPIIGaze datasets.

Appearance-Based Gaze Estimation Using Dilated-Convolutions

Electroencephalogram (EEG) signals classification plays a crucial role in brain computer interfaces (BCIs) system. However, the inherent complex properties of EEG signals make it challenging to get them analyzed and modeled. In this paper, a novel method based on conditional empirical mode decomposition (CEMD) and one-dimensional multi-scale convolutional neural network (1DMSCNN) is proposed to recognize motor imagery (MI) EEG signals. In the CEMD algorithm, the correlation coefficient between the original EEG signal and each intrinsic modal component (IMF) is used as the first condition to select IMFs, and the relative energy occupancy rates between the IMFs are the second condition. The CEMD algorithm is applied to remove the noise of EEG signals. Then, an EEG signals combination method is proposed to encode event-related synchronization/de-synchronization (ERS/ERD) information between the channels. Finally, a model called 1DMSCNN is built to classify the processed EEG signals. The proposed method is applied to the dataset collected in our laboratory and BCI competition IV dataset 2b. The results indicate that the proposed method can achieve higher accuracy for EEG signals classification, compared with other state-of-the-art works. In addition, the proposed algorithm is applied to the online recognition of EEG signals, a BCI system that directly interacts with brain and wheelchair is designed and implemented. This system can directly command wheelchair to turn left and right through EEG signals. The online experimental results indicate that the designed intelligent wheelchair system is a feasible BCI application. It verifies the proposed algorithm can be used in expert and intelligent systems. Our method can provide a stimulus to the development of human-robot interaction.

Motor imagery EEG recognition based on conditional optimization empirical mode decomposition and multi-scale convolutional neural network

A Brain-Computer Interface (BCI) acts as a communication mechanism using brain signals to control external devices The generation of such signals is sometimes independent of the nervous system, such as in Passive BCI This is majorly beneficial for those who have severe motor disabilities Traditional BCI systems have been dependent only on brain signals recorded using Electroencephalography (EEG) and have used a rule-based translation algorithm to generate control commands However, the recent use of multi-sensor data fusion and machine learning-based translation algorithms has improved the accuracy of such systems This paper discusses various BCI applications such as tele-presence, grasping of objects, navigation, etc that use multi-sensor fusion and machine learning to control a humanoid robot to perform a desired task The paper also includes a review of the methods and system design used in the discussed applications

https://www.mdpi.com/1424-8220/20/13/3620/pdf

Brain-Computer Interface-Based Humanoid Control: A Review.

The automatic detection of eye positions, their temporal consistency, and their mapping into a line of sight in the real world (to find where a person is looking at) is reported in the scientific literature as gaze tracking. This has become a very hot topic in the field of computer vision during the last decades, with a surprising and continuously growing number of application fields. A very long journey has been made from the first pioneering works, and this continuous search for more accurate solutions process has been further boosted in the last decade when deep neural networks have revolutionized the whole machine learning area, and gaze tracking as well. In this arena, it is being increasingly useful to find guidance through survey/review articles collecting most relevant works and putting clear pros and cons of existing techniques, also by introducing a precise taxonomy. This kind of manuscripts allows researchers and technicians to choose the better way to move towards their application or scientific goals. In the literature, there exist holistic and specifically technological survey documents (even if not updated), but, unfortunately, there is not an overview discussing how the great advancements in computer vision have impacted gaze tracking. Thus, this work represents an attempt to fill this gap, also introducing a wider point of view that brings to a new taxonomy (extending the consolidated ones) by considering gaze tracking as a more exhaustive task that aims at estimating gaze target from different perspectives: from the eye of the beholder (first-person view), from an external camera framing the beholder's, from a third-person view looking at the scene where the beholder is placed in, and from an external view independent from the beholder.

/pdf/when-i-look-into-your-eyes-a-survey-on-computer-vision-46upjfr5g5.pdf

When I Look into Your Eyes: A Survey on Computer Vision Contributions for Human Gaze Estimation and Tracking

We describe a hybrid brain computer interface that integrates information from a four-class motor imagery based EEG classifier with information about gaze trajectories from an eye tracker. The novel aspect of this system is that no explicit gaze behavior is required of the user. Rather, the natural gaze behavior of the user integrated probabilistically to smooth the noisy classification results from the motor imagery based EEG. The goal is to provide for a more natural interaction with the BCI system than if gaze were used as an explicit command signal, as is commonly done. Our results on a 2D cursor control task show that integration of gaze information significantly improves task completion accuracy and reduces task completion time. In particular, our system achieves over 80% target completion accuracy on a cursor control task requiring guidance to one of 12 targets.

Hybrid Brain Computer Interface via Bayesian integration of EEG and eye gaze

In building human robot interaction systems, it would be helpful to understand how humans collaborate, and in particular, how humans use others' gaze behavior to estimate their intent. Here we studied the use of gaze in a collaborative assembly task, where a human user assembled an object with the assistance of a human helper. We found that the being aware of the partner's gaze significantly improved collaboration efficiency. Task completion times were much shorter when gaze communication was available, than when it was blocked. In addition, we found that the user's gaze was more likely to lie on the object of interest in the gaze-aware case than the gaze-blocked case. In the context of human-robot collaboration systems, our results suggest that gaze data in the period surrounding verbal requests will be more informative and can be used to predict the target object.

Gaze awareness improves collaboration efficiency in a collaborative assembly task

This paper addresses the problem of estimating gaze location in the 3D environment using a remote eye tracker. Instead of relying only on data provided by the eye tracker, we investigate how to integrate gaze direction with the point-cloud-based representation of the scene provided by a Kinect sensor. The algorithm first combines the gaze vectors for the two eyes provided by the eye tracker into a single gaze vector emanating from a point in between the two eyes. The gaze target in the three dimensional environment is then identified by finding the point in the 3D point cloud that is closest to the gaze vector. Our experimental results demonstrate that the estimate of the gaze target location provided by this method is significantly better than that provided when considering gaze information alone. It is also better than two other methods for integrating point cloud information: (1) finding the 3D point closest to the gaze location as estimated by triangulating the gaze vectors from the two eyes, and (2) finding the 3D point with smallest average distance to the two gaze vectors considered individually. The proposed method has an average error of 1.7 cm in a workspace of 25 × 23 × 24 cm located at a distance of 60 cm from the user.

Using point cloud data to improve three dimensional gaze estimation

Deep neural networks have demonstrated superior perfor- mance on appearance-based gaze estimation tasks. However, due to variations in person, illuminations, and background, performance degrades dramatically when applying the model to a new domain. In this paper, we discover an interesting gaze jitter phenomenon in cross-domain gaze estimation, i.e. , the gaze predictions of two similar images can be severely deviated in target domain. This is closely related to cross-domain gaze estimation tasks, but surprisingly, it has not been noticed yet previously. Therefore, we innovatively propose to utilize the gaze jitter to analyze and optimize the gaze domain adaptation task. We ﬁnd that the high-frequency component (HFC) is an important factor that leads to jitter. Based on this discovery, we add high-frequency components to input images using the adversarial attack and employ contrastive learning to encourage the model to obtain similar representa-tions between original and perturbed data, which reduces the impacts of HFC. We evaluate the proposed method on four cross-domain gaze estimation tasks, and experimental results demonstrate that it signiﬁcantly reduces the gaze jitter and improves the gaze estimation performance in target domains.

pdf/jitter-does-matter-adapting-gaze-estimation-to-new-domains-mf3078qs.pdf

Jitter Does Matter: Adapting Gaze Estimation to New Domains

Remote photoplethysmography (rPPG) is a non-contact technology that can estimate heart rate using facial video and holds significant potential for health monitoring. Despite the latest deep learning-based rPPG approaches can predict high-quality rPPG signal under similar scenarios, these methods often suffer from degraded performance when encountering variations in subjects, environments, or illumination conditions in target domains. To address this challenge, we propose an uncertainty-guided self-training approach that leverages model uncertainty and periodic priors to enhance generalization across different domains without requiring labels in the target domain. We iteratively update the model using pseudo-labels generated from its own predictions on unlabelled data in the target domain, with varying confidence levels informed by the model’s uncertainty estimation. To achieve this, we modify a standard Convolutional Neural Network (CNN) into a Bayesian Neural Network (BNN) for uncertainty estimation, which guides the assignment of pseudo-labels with varying confidence levels. By employing the adversarially learned periodic priors of rPPG signals shared across domains as regularization terms, we further stabilize the model adaptation process. We evaluate the proposed method on two public datasets (PURE and UBFC-rPPG) across five cross-domain tasks. Experimental results demonstrate improved performance over the baselines, with gains ranging from 60.5% to 97.2%, outperforming existing methods in generalization performance for rPPG-based heart rate measurement.

Towards Practical Facial Video-based Remote Heart Rate Estimation via Cross Domain rPPG Adaptation

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Haofei Wang

Author Tools

Chat about Author