Enabling touch gesture sensing on all surfaces of the mobile device, not limited to the touchscreen area, leads to new user interaction experiences. In this paper, we propose VSkin, a system that supports fine-grained gesture-sensing on the back of mobile devices based on acoustic signals. VSkin utilizes both the structure-borne sounds, i.e., sounds propagating through the structure of the device, and the air-borne sounds, i.e., sounds propagating through the air, to sense finger tapping and movements. By measuring both the amplitude and the phase of each path of sound signals, VSkin detects tapping events with an accuracy of 99.65% and captures finger movements with an accuracy of 3.59mm.

VSkin: Sensing Touch Gestures on Surfaces of Mobile Devices Using Acoustic Signals

We design a system, SolarGest, which can recognize hand gestures near a solar-powered device by analyzing the patterns of the photocurrent. SolarGest is based on the observation that each gesture interferes with incident light rays on the solar panel in a unique way, leaving its distinguishable signature in harvested photocurrent. Using solar energy harvesting laws, we develop a model to optimize design and usage of SolarGest. To further improve the robustness of SolarGest under non-deterministic operating conditions, we combine dynamic time warping with Z-score transformation in a signal processing pipeline to pre-process each gesture waveform before it is analyzed for classification. We evaluate SolarGest with both conventional opaque solar cells as well as emerging see-through transparent cells. Our experiments with 6,960 gesture samples for 6 different gestures reveal that even with transparent cells, SolarGest can detect 96% of the gestures while consuming 44% less power compared to light sensor based systems.

pdf/solargest-ubiquitous-and-battery-free-gesture-recognition-1wyhg98lhn.pdf

SolarGest: Ubiquitous and Battery-free Gesture Recognition using Solar Cells

There is a growing interest in the scientific community to develop techniques for humans to communicate with the computing that is embedding into our environments. Researchers are already exploring ubiquitous modalities, such as radio frequency signals, to develop gesture recognition systems. In this paper, we explore another such modality, namely ambient light, and develop LiGest, an ambient light based gesture recognition system. The key property of LiGest is that it is agnostic to lighting conditions, position and orientation of user, and who performs the gestures. The general idea behind LiGest is that when a user performs different gestures, the shadows of the user move in unique patterns. LiGest first learns these patterns using training samples and then recognizes unknown samples by matching them with the learnt patterns. To capture these patterns, LiGest uses a grid of light sensors deployed on floor. While the general idea behind LiGest seems straightforward, it is actually very challenging to put it into practice because the intensity, size, and number of shadows of a user are not fixed and depend highly on the position and orientation of a user as well as on the intensity, position, and number of light sources. We developed a prototype of LiGest using commercially available light sensors and extensively evaluated it with the help of 20 volunteers. Our results show that LiGest achieves an average accuracy of 96.36% across all volunteers.

https://dl.acm.org/doi/pdf/10.1145/3191772

Gesture Recognition Using Ambient Light

Design Recommendations for Intelligent Tutoring Systems: Volume 4-Domain Modeling

Despite decades of research, there is yet no general rapid prototyping recognizer for dynamic gestures that can be trained with few samples, work with continuous data, and achieve high accuracy that is also modality-agnostic. To begin to solve this problem, we describe a small suite of accessible techniques that we collectively refer to as the Jackknife gesture recognizer. Our dynamic time warping based approach for both segmented and continuous data is designed to be a robust, go-to method for gesture recognition across a variety of modalities using only limited training samples. We evaluate pen and touch, Wii Remote, Kinect, Leap Motion, and sound-sensed gesture datasets as well as conduct tests with continuous data. Across all scenarios we show that our approach is able to achieve high accuracy, suggesting that Jackknife is a capable recognizer and good first choice for many endeavors.

https://par.nsf.gov/servlets/purl/10026269

Jackknife: A Reliable Recognizer with Few Samples and Many Modalities

Intelligent Tutoring Systems (ITSs) have been shown to be effective in training a variety of military tasks. However, these systems are often limited to laboratory settings on standard PCs and laptops which focus on exercising cognitive skills (e.g., decision-making and problem solving) and may potentially limit the learning and retention of the dismounted Soldiers and Marines training to master physical tasks. Augmented reality presents the possibility of combining intelligent tutoring with hands-on applications in realistic physical environments. In this paper, we examine the use of an augment-reality based adaptive tutoring system for instruction in the wild, locations where no formal training infrastructure is present, and identify the challenges that arise when developing such a system. We began the transition from desktop tutoring to the wild by exploring an existing real life mockup of a market scene along with low cost commercial-off-the-shelf devices (e.g., HMDs coupled with depth cameras) and a 3D model of the environment. The goal of our canning approach is to use “human in the loop” 3D scene acquisition via augmented reality so that the scene can be scanned efficiently and with complete coverage. Using this 3D model, intelligent tutoring systems can adaptively manage instruction while being aware of the physical and augmented objects in the scenario. Furthermore, with this awareness of the physical environment, we hope to provide augmented effects and objects (e.g., virtual humans) that register to the physical environment and respond realistically to interactions with the trainee. We also explored developing a training scenario for evaluation of our system that is made to work with emerging low-cost commercial augmented reality devices (e.g., Epson Moverio). Our approach examines the merging of intelligent tutoring with augmented reality to be used for hands-on immersive training of psychomotor tasks in a setting beyond the typical desktop tutoring session.

/pdf/using-augmented-reality-to-tutor-military-tasks-in-the-wild-588zvab2iv.pdf

Using Augmented Reality to Tutor Military Tasks in the Wild

We constructed an acoustic, gesture-based recognition system called Multiwave, which leverages the Doppler Effect to translate multidimensional movements into user interface commands. Our system only requires the use of two speakers and a microphone to be operational. Since these components are already built in to most end user systems, our design makes gesture-based input more accessible to a wider range of end users. By generating a known high frequency tone from multiple speakers and detecting movement using changes in the sound waves, we are able to calculate a Euclidean representation of hand velocity that is then used for more natural gesture recognition and thus, more meaningful interaction mappings. We present the results of a user study of Multiwave to evaluate recognition rates for different gestures and report accuracy rates comparable to or better than the current state of the art. We also report subjective user feedback and some lessons learned from our system that provide additional insight for future applications of multidimensional gesture recognition.

Multiwave: Doppler Effect Based Gesture Recognition in Multiple Dimensions

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Conner Brooks

Author Tools

Chat about Author

Papers

Using Augmented Reality to Tutor Military Tasks in the Wild

Multiwave: Doppler Effect Based Gesture Recognition in Multiple Dimensions