In this paper, we present our work in building technologies for natural multimodal human-robot interaction. We present our systems for spontaneous speech recognition, multimodal dialogue processing, and visual perception of a user, which includes localization, tracking, and identification of the user, recognition of pointing gestures, as well as the recognition of a person's head orientation. Each of the components is described in the paper and experimental results are presented. We also present several experiments on multimodal human-robot interaction, such as interaction using speech and gestures, the automatic determination of the addressee during human-human-robot interaction, as well on interactive learning of dialogue strategies. The work and the components presented here constitute the core building blocks for audiovisual perception of humans and multimodal human-robot interaction used for the humanoid robot developed within the German research project (Sonderforschungsbereich) on humanoid cooperative robots.

Enabling Multimodal Human–Robot Interaction for the Karlsruhe Humanoid Robot

A dialogue approach to learning object descriptions and semantic categories

This paper describes the video-based face recognition evaluation performed under the CHIL project and the systems that participated to it, along with the obtained first year results. The evaluation methodology comprises a specially built database of videos and an evaluation protocol. Two complete automatic face detection and recognition systems from two academic institutions participated to the evaluation. For comparison purposes, a baseline system is also developed using well-known methods for face detection and recognition.

Video-Based Face Recognition Evaluation in the CHIL Project - Run 1.

Purpose – This paper aims to give an overview of a dialogue manager and recent experiments with multimodal human‐robot dialogues.Design/methodology/approach – The paper identifies requirements and solutions in the design of a human‐robot interface. The paper presents essential techniques for a humanoid robot in a household environment and describes their application to representative interaction scenarios that are based on standard situations for a humanoid robot in a household environment. The presented dialogue manager has been developed within the German collaborative research center SFB‐588 on “Humanoid Robots – Learning and Cooperating Multimodal Robots”. The dialogue system is embedded in a multimodal perceptual system of the humanoid robot developed within this project. The implementation of the dialogue manager is geared to requirements found in the explored scenarios. The algorithms include multimodal fusion, reinforcement learning, knowledge acquisition and tight coupling of dialogue manager and...

A dialogue manager for multimodal human‐robot interaction and learning of a humanoid robot

Service robot has received enormous attention with rapid development of advanced technology in recent years, and it is endowed with the capabilities of performing human-robot interaction (HRI). We construct a sensor fusion based system to integrate the information from both sensors by using a data association approach - Covariance Intersection (CI). It will be used to increase the robustness and reliability of HRI in the real world environment. In this paper, we propose a Behavior System for analyzing human features and classifying the behavior by the crucial information from sensor fusion system. The system is used to infer the human behavioral intentions, and also allow the robot to perform more natural and intelligent interaction. We apply a spatial model based on proxemics rules to our robot, and design a behavioral intention inference strategy. Furthermore, the robot will make the corresponding reaction in accordance with the identified behavioral intention.

Sensor fusion based human detection and tracking system for human-robot interaction

Acquiring knowledge about persons is a key functionality for humanoid robots. In a natural environment, the robot not only interacts with different people who he recognizes and who he knows. He will also have to interact with unknown persons, and by acquiring information about them, the robot can memorize these persons and provide extended personalized services. Today, researchers build systems to recognize a person's face, voice and other features. Most of them depend on precollected data. We think that with the given technology it is about time to build a system that collects data autonomously and thus gets to know and learns to recognize persons completely on its own.

This paper describes the integration of different perceptual and dialog components and their individual functionality to build a robot that can contact persons, learns their names, and learns to recognize them in future encounters.

A robot learns to know people: first contacts of a robot

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Christoph Schaa

Author Tools

Chat about Author

Papers

A robot learns to know people: first contacts of a robot