We present a suite of tools to model a robot, its sensors, and the surrounding environment in VR, with the goal of collecting training data for real-world robots. The virtual robot observes a rigged avatar created in our photogrammetry facility and embodying a VR user. We are particularly interested in verbal human/robot interactions, which can be combined with the robot’s sensor data for grounded language learning. Because virtual scenes, tasks, and robots are easily reconfigured compared to their physical analogs, our approach proves extremely versatile in preparing a wide range of robot scenarios for an array of use cases.

A Simulator for Human-Robot Interaction in Virtual Reality

Even with impressive advances in application specific models, we still lack knowledge about how to build a model that can learn in a human-like way and do multiple tasks. To learn in a human-like way, we need to provide a diverse experience that is comparable to human's. In this paper, we introduce our ongoing effort to build a simulated environment for developmental robotics (SEDRo). SEDRo provides diverse human experiences ranging from those of a fetus to a 12th month old. A series of simulated tests based on developmental psychology will be used to evaluate the progress of a learning model. We anticipate SEDRo to lower the cost of entry and facilitate research in the developmental robotics community.

SEDRo: A Simulated Environment for Developmental Robotics

There is increasing interest in using robots in simulation to understand and improve human-robot interaction (HRI). At the same time, the use of simulated settings to gather training data promises to help address a major data bottleneck in allowing robots to take advantage of powerful machine learning approaches. In this paper, we describe a prototype system that combines the robot operating system (ROS), the simulator Gazebo, and the Unity game engine to create human-robot interaction scenarios. A person can engage with the scenario using a monitor wall, allowing simultaneous collection of realistic sensor data and traces of human actions.

https://dl.acm.org/doi/pdf/10.1145/3306214.3338546

Learning from human-robot interactions in modeled scenes

Even with impressive advances in application-specific models, we still lack knowledge about how to build a model that can learn in a human-like way and do multiple tasks. To learn in a human-like way, we need to provide a diverse experience that is comparable to humans. In this paper, we introduce our ongoing effort to build a simulated environment for developmental robotics (SEDRo). SEDRo provides diverse human experiences ranging from those of a fetus to a 12th-month-old. A series of simulated tests based on developmental psychology will be used to evaluate the progress of a learning model. We anticipate SEDRo to lower the cost of entry and facilitate research in the developmental robotics community.

SEDRo: A Simulated Environment for Developmental Robotics.

Photogrammetry is a science dealing with obtaining reliable information about physical objects using their imagery description. Recent advancements in the development of Virtual Reality (VR) can help to unlock the full potential offered by the digital 3D-reality models generated using the state-of-art photogrammetric technologies. These models are becoming a viable alternative for providing high-quality content for such immersive environment. Simultaneously, their analyses in VR could bring added-value to professionals working in various engineering and non-engineering settings and help in extracting useful information about physical objects. However, there is little research published to date on feasible interaction methods in the VR-based systems augmented with the 3D photogrammetric models, especially concerning gestural input interfaces. Consequently, this paper presents the PhotoTwinVR -- an immersive, gesture-controlled system for manipulation and inspection of 3D photogrammetric models of physical objects in VR. Our system allows the user to perform basic engineering operations on the model subjected to the off-line inspection process. An observational study with a group of three domain-expert participants was completed to verify its feasibility. The system was populated with a 3D photogrammetric model of an existing pipe-rack generated using a commercial software package. The participants were asked to carry out a survey measurement of the object using the measurement toolbox offered by PhotoTwinVR. The study revealed a potential of such immersive tool to be applied in practical real-words cases of off-line inspections of pipelines.

PhotoTwinVR: An Immersive System for Manipulation, Inspection and Dimension Measurements of the 3D Photogrammetric Models of Real-Life Structures in Virtual Reality

Collecting data in robotics, especially human-robot interactions, traditionally requires a physical robot in a prepared environment, that presents substantial scalability challenges. First, robots provide many possible points of system failure, while the availability of human participants is limited. Second, for tasks such as language learning, it is important to create environments that provide interesting’ varied use cases. Traditionally, this requires prepared physical spaces for each scenario being studied. Finally, the expense associated with acquiring robots and preparing spaces places serious limitations on the reproducible quality of experiments. We therefore propose a novel mechanism for using virtual reality to simulate robotic sensor data in a series of prepared scenarios. This allows for a reproducible dataset that other labs can recreate using commodity VR hardware. We demonstrate the effectiveness of this approach with an implementation that includes a simulated physical context, a reconstruction of a human actor, and a reconstruction of a robot. This evaluation shows that even a simple “sandbox” environment allows us to simulate robot sensor data, as well as the movement (e.g., view-port) and speech of humans interacting with the robot in a prescribed scenario.

/pdf/virtual-reality-and-photogrammetry-for-improved-4mc9b59wyp.pdf

Virtual Reality and Photogrammetry for Improved Reproducibility of Human-Robot Interaction Studies

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Don EnQel

Author Tools

Chat about Author

Papers

Virtual Reality and Photogrammetry for Improved Reproducibility of Human-Robot Interaction Studies