In this paper, we present a multimodal emotion recognition framework called EmotionMeter that combines brain waves and eye movements To increase the feasibility and wearability of EmotionMeter in real-world applications, we design a six-electrode placement above the ears to collect electroencephalography (EEG) signals We combine EEG and eye movements for integrating the internal cognitive states and external subconscious behaviors of users to improve the recognition accuracy of EmotionMeter  The experimental results demonstrate that modality fusion with multimodal deep neural networks can significantly enhance the performance compared with a single modality, and the best mean accuracy of 8511% is achieved for four emotions (happy, sad, fear, and neutral) We explore the complementary characteristics of EEG and eye movements for their representational capacities and identify that EEG has the advantage of classifying happy emotion, whereas eye movements outperform EEG in recognizing fear emotion To investigate the stability of EmotionMeter over time, each subject performs the experiments three times on different days EmotionMeter obtains a mean recognition accuracy of 7239% across sessions with the six-electrode EEG and eye movement features These experimental results demonstrate the effectiveness of EmotionMeter within and between sessions

EmotionMeter: A Multimodal Framework for Recognizing Human Emotions

Background: Digital mental health interventions (DMHIs), which deliver mental health support via technologies such as mobile apps, can increase access to mental health support, and many studies have demonstrated their effectiveness in improving symptoms. However, user engagement varies, with regard to a user’s uptake and sustained interactions with these interventions.
Objective: This systematic review aims to identify common barriers and facilitators that influence user engagement with DMHIs.
Methods: A systematic search was conducted in the SCOPUS, PubMed, PsycINFO, Web of Science, and Cochrane Library databases. Empirical studies that report qualitative and/or quantitative data were included.
Results: A total of 208 articles met the inclusion criteria. The included articles used a variety of methodologies, including interviews, surveys, focus groups, workshops, field studies, and analysis of user reviews. Factors extracted for coding were related to the end user, the program or content offered by the intervention, and the technology and implementation environment. Common barriers included severe mental health issues that hampered engagement, technical issues, and a lack of personalization. Common facilitators were social connectedness facilitated by the intervention, increased insight into health, and a feeling of being in control of one’s own health.
Conclusions: Although previous research suggests that DMHIs can be useful in supporting mental health, contextual factors are important determinants of whether users actually engage with these interventions. The factors identified in this review can provide guidance when evaluating DMHIs to help explain and understand user engagement and can inform the design and development of new digital interventions.
Trial Registration:

/pdf/barriers-to-and-facilitators-of-user-engagement-with-digital-4p4iga4we6.pdf

Barriers to and Facilitators of User Engagement With Digital Mental Health Interventions: Systematic Review

From an accessibility perspective, voice-controlled, home-based intelligent personal assistants (IPAs) have the potential to greatly expand speech interaction beyond dictation and screen reader output. To examine the accessibility of off-the-shelf IPAs (e.g., Amazon Echo) and to understand how users with disabilities are making use of these devices, we conducted two exploratory studies. The first, broader study is a content analysis of 346 Amazon Echo reviews that include users with disabilities, while the second study more specifically focuses on users with visual impairments, through interviews with 16 current users of home-based IPAs. Findings show that, although some accessibility challenges exist, users with a range of disabilities are using the Amazon Echo, including for unexpected cases such as speech therapy and support for caregivers. Richer voice-based applications and solutions to support discoverability would be particularly useful to users with visual impairments. These findings should inform future work on accessible voice-based IPAs.

"Accessibility Came by Accident": Use of Voice-Controlled Intelligent Personal Assistants by People with Disabilities

Head and eye movement can be leveraged to improve the user's interaction repertoire for wearable displays. Head movements are deliberate and accurate, and provide the current state-of-the-art pointing technique. Eye gaze can potentially be faster and more ergonomic, but suffers from low accuracy due to calibration errors and drift of wearable eye-tracking sensors. This work investigates precise, multimodal selection techniques using head motion and eye gaze. A comparison of speed and pointing accuracy reveals the relative merits of each method, including the achievable target size for robust selection. We demonstrate and discuss example applications for augmented reality, including compact menus with deep structure, and a proof-of-concept method for on-line correction of calibration drift.

Pinpointing: Precise Head- and Eye-Based Target Selection for Augmented Reality

Background: The global shortage of mental health workers has prompted the utilization of technological advancements, such as chatbots, to meet the needs of people with mental health conditions. Chatbots are systems that are able to converse and interact with human users using spoken, written, and visual language. While numerous studies have assessed the effectiveness and safety of using chatbots in mental health, no reviews have pooled the results of those studies.
Objective: This study aimed to assess the effectiveness and safety of using chatbots to improve mental health through summarizing and pooling the results of previous studies.
Methods: A systematic review was carried out to achieve this objective. The search sources were 7 bibliographic databases (eg, MEDLINE, EMBASE, PsycINFO), the search engine “Google Scholar,” and backward and forward reference list checking of the included studies and relevant reviews. Two reviewers independently selected the studies, extracted data from the included studies, and assessed the risk of bias. Data extracted from studies were synthesized using narrative and statistical methods, as appropriate.
Results: Of 1048 citations retrieved, we identified 12 studies examining the effect of using chatbots on 8 outcomes. Weak evidence demonstrated that chatbots were effective in improving depression, distress, stress, and acrophobia. In contrast, according to similar evidence, there was no statistically significant effect of using chatbots on subjective psychological wellbeing. Results were conflicting regarding the effect of chatbots on the severity of anxiety and positive and negative affect. Only two studies assessed the safety of chatbots and concluded that they are safe in mental health, as no adverse events or harms were reported.
Conclusions: Chatbots have the potential to improve mental health. However, the evidence in this review was not sufficient to definitely conclude this due to lack of evidence that their effect is clinically important, a lack of studies assessing each outcome, high risk of bias in those studies, and conflicting results for some outcomes. Further studies are required to draw solid conclusions about the effectiveness and safety of chatbots.
Trial Registration: PROSPERO International Prospective Register of Systematic Reviews CRD42019141219; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42019141219

Effectiveness and Safety of Using Chatbots to Improve Mental Health: Systematic Review and Meta-Analysis

For eye tracking to become a ubiquitous part of our everyday interaction with computers, we first need to understand its limitations outside rigorously controlled labs, and develop robust applications that can be used by a broad range of users and in various environments. Toward this end, we collected eye tracking data from 80 people in a calibration-style task, using two different trackers in two lighting conditions. We found that accuracy and precision can vary between users and targets more than six-fold, and report on differences between lighting, trackers, and screen regions. We show how such data can be used to determine appropriate target sizes and to optimize the parameters of commonly used filters. We conclude with design recommendations and examples how our findings and methodology can inform the design of error-aware adaptive applications.

/pdf/toward-everyday-gaze-input-accuracy-and-precision-of-eye-2zfir87r6h.pdf

Toward Everyday Gaze Input: Accuracy and Precision of Eye Tracking and Implications for Design

Mental health disorders are a leading cause of disability worldwide. Although evidence-based psychotherapy is effective, engagement from such programs can be low. Mobile apps have the potential to help engage and support people in their therapy. We developed Pocket Skills, a mobile web app based on Dialectical Behavior Therapy (DBT). Pocket Skills teaches DBT via a conversational agent modeled on Marsha Linehan, who developed DBT. We examined the feasibility of Pocket Skills in a 4-week field study with 73 individuals enrolled in psychotherapy. After the study, participants reported decreased depression and anxiety and increased DBT skills use. We present a model based on qualitative findings of how Pocket Skills supported DBT. Pocket Skills helped participants engage in their DBT and practice and implement skills in their environmental context, which enabled them to see the results of using their DBT skills and increase their self-efficacy. We discuss the design implications of these findings for future mobile mental health systems.

/pdf/pocket-skills-a-conversational-mobile-web-app-to-support-3qskbe01vo.pdf

Pocket Skills: A Conversational Mobile Web App To Support Dialectical Behavioral Therapy

Gaze-based alternative and augmentative communication (AAC) devices provide users with neuromuscular diseases the ability to communicate with other people through only the movement of their eyes. These devices suffer from slow input, causing a host of communication breakdowns to occur during face-to-face conversations. These breakdowns lead to decreased user autonomy, conversation quality, and communication partner engagement. Attempts to improve communication through these devices has mainly focused on throughput and rate enhancement, though this has only attained meager results to date. In this work, we address this issue from the top down by considering AAC devices as a form of groupware and designing interactions around this groupware that facilitate better conversations for all involved communicators. We first present qualitative findings on issues with gaze-based AAC and end-user communication preferences; we identify several design guidelines for improving these systems and then present AACrobat, a system that embodies these guidelines and introduces novel interactions by extending gaze-based AAC devices with a mobile companion app. Finally, we present early feedback on AACrobat through three case studies of users with ALS.

/pdf/aacrobat-using-mobile-devices-to-lower-communication-3s256cr90g.pdf

AACrobat: Using Mobile Devices to Lower Communication Barriers and Provide Autonomy with Gaze-Based AAC

Augmentative and alternative communication (AAC) devices are a critical technology for people with disabilities that affect their speech. One challenge with AAC systems is their inability to portray aspects of nonverbal communication that typically accent, complement, regulate, or substitute for verbal speech. In this paper, we explore the design space of awareness displays that can supplement AAC devices, considering their output features and their effects on the perceptions of interlocutors. Through designing prototypes and getting feedback on our designs from people with ALS, their primary caregivers, and other communication partners, we consider (1) the consistent tensions that arose between abstractness and clarity in meaning for these designs and (2) the ways in which these designs can further mark users as "other." Overall, we contribute a generative understanding of designing AAC awareness displays to augment and contextualize communication.

/pdf/exploring-the-design-space-of-aac-awareness-displays-20zkof982u.pdf

Exploring the Design Space of AAC Awareness Displays

Remnance of Form is an interactive installation that explores the dynamic tension between an object and its shadow. By fusing light, projection, and motion technologies, the shadow can now detach itself from its former role. This creates a new narrative that challenges our perception of reality, what's real and what's not.

Remnance of Form: Altered Reflection of Physical Reality

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Ann Paradiso

Author Tools

Chat about Author