Large-language models (LLMs) hold significant promise in improving human-robot interaction, offering advanced conversational skills and versatility in managing diverse, open-ended user requests in various tasks and domains. Despite the potential to transform human-robot interaction, very little is known about the distinctive design requirements for utilizing LLMs in robots, which may differ from text and voice interaction and vary by task and context. To better understand these requirements, we conducted a user study (n = 32) comparing an LLM-powered social robot against text- and voice-based agents, analyzing task-based requirements in conversational tasks, including choose, generate, execute, and negotiate. Our findings show that LLM-powered robots elevate expectations for sophisticated non-verbal cues and excel in connection-building and deliberation, but fall short in logical communication and may induce anxiety. We provide design implications both for robots integrating LLMs and for fine-tuning LLMs for use with robots.

Understanding Large-Language Model (LLM)-powered Human-Robot Interaction

Abstract This work aims to provide initial guidelines towards developing companion robots with large language models (LLMs) to be part of everyday lives of older adults. Using iterative participatory design (co-design) approaches, we analyze the challenges of applying LLMs for multi-modal open-domain dialogue, deriving from older adults' (one-to-one) interactions with a personalized companion robot, built on Furhat robot with GPT-3.5. An initial study with 6 Swedish-speaking older adults (65 and older) showed that the robot frequently interrupted the users, responded slowly and repetitively, engaged in superficial conversations, and caused a barrier in the interaction due to foreign language (English). Upon incremental technical developments to address these issues, participatory design workshops were conducted with 28 Swedish-speaking older adults. While the interactions (in Swedish) were smoother, less disrupted, and more varied in topics and responses, further challenges were observed due to hallucinations and obsolete information, and disengagement cues, causing frustration, confusion, and worry. 

/pdf/between-reality-and-delusion-challenges-of-applying-large-12h233hs.pdf

Between Reality and Delusion: Challenges of Applying Large Language Models to Companion Robots for Open-Domain Dialogues with Older Adults

Companion robots are aimed to mitigate loneliness and social isolation among older adults by providing social and emotional support in their everyday lives. However, older adults’ expectations of conversational companionship might substantially differ from what current technologies can achieve, as well as from other age groups like young adults. Thus, it is crucial to involve older adults in the development of conversational companion robots to ensure that these devices align with their unique expectations and experiences. The recent advancement in foundation models, such as large language models, has taken a significant stride toward fulfilling those expectations, in contrast to the prior literature that relied on humans controlling robots (i.e., Wizard of Oz) or limited rule-based architectures that are not feasible to apply in the daily lives of older adults. Consequently, we conducted a participatory design (co-design) study with 28 older adults, demonstrating a companion robot using a large language model (LLM), and design scenarios that represent situations from everyday life. The thematic analysis of the discussions around these scenarios shows that older adults expect a conversational companion robot to engage in conversation actively in isolation and passively in social settings, remember previous conversations and personalize, protect privacy and provide control over learned data, give information and daily reminders, foster social skills and connections, and express empathy and emotions. Based on these findings, this article provides actionable recommendations for designing conversational companion robots for older adults with foundation models, such as LLMs and vision-language models, which can also be applied to conversational robots in other domains. 

pdf/recommendations-for-designing-conversational-companion-5fi3xggf6a.pdf

Recommendations for designing conversational companion robots with older adults through foundation models

Abstract In recent years, the integration of large language models (LLMs) has revolutionized the field of robotics, enabling robots to communicate, understand, and reason with human-like proficiency. This paper explores the multifaceted impact of LLMs on robotics, addressing key challenges and opportunities for leveraging these models across various domains. By categorizing and analyzing LLM applications within core robotics elements—communication, perception, planning, and control—we aim to provide actionable insights for researchers seeking to integrate LLMs into their robotic systems. Our investigation focuses on LLMs developed post-GPT-3.5, primarily in text-based modalities while also considering multimodal approaches for perception and control. We offer comprehensive guidelines and examples for prompt engineering, facilitating beginners’ access to LLM-based robotics solutions. Through tutorial-level examples and structured prompt construction, we illustrate how LLM-guided enhancements can be seamlessly integrated into robotics applications. This survey serves as a roadmap for researchers navigating the evolving landscape of LLM-driven robotics, offering a comprehensive overview and practical guidance for harnessing the power of language models in robotics development. 

A survey on integration of large language models with intelligent robots

Throughout our lives, we interact daily in conversations with our friends and family, covering a wide range of topics, known as open-domain dialogue. As we age, these interactions may diminish due to changes in social and personal relationships, leading to loneliness in older adults. Conversational companion robots can alleviate this issue by providing daily social support. Large language models (LLMs) offer flexibility for enabling open-domain dialogue in these robots. However, LLMs are typically trained and evaluated on textual data, while robots introduce additional complexity through multi-modal interactions, which has not been explored in prior studies. Moreover, it is crucial to involve older adults in the development of robots to ensure alignment with their needs and expectations. Correspondingly, using iterative participatory design approaches, this paper exposes the challenges of integrating LLMs into conversational robots, deriving from 34 Swedish-speaking older adults’ (one-to-one) interactions with a personalized companion robot, built on Furhat robot with GPT-\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$-$$\end{document}3.5. These challenges encompass disruptions in conversations, including frequent interruptions, slow, repetitive, superficial, incoherent, and disengaging responses, language barriers, hallucinations, and outdated information, leading to frustration, confusion, and worry among older adults. Drawing on insights from these challenges, we offer recommendations to enhance the integration of LLMs into conversational robots, encompassing both general suggestions and those tailored to companion robots for older adults.

Between reality and delusion: challenges of applying large language models to companion robots for open-domain dialogues with older adults

Robots may be able to significantly assist older adults through making activity recommendations. Prior research suggests that gender and age of a robot’s voice may affect how people respond to such recommendations, but few studies have explored how a robot’s voice is perceived by older adults, and whether their perceptions differ across cultures. We conducted a survey study with older adult participants (aged 65+) in the U.S. (N=225) and Japan (N=466), asking them to evaluate a humanoid robot speaking with three different voices (male, female, child). After seeing a video of a robot making recommendations, participants rated the fit of the voice to the robot, its sociality (via the Robotic Social Attributes Scale - RoSAS), and their willingness to use the robot in various contexts. We discovered that robot’s social attributes and participants’ culture impacted willingness to use the robot in both countries. Having positive social attributes and lower negative attributes increases willingness to use the robot. The U.S. older adults preferred the adult robot voices, had more positive social attributes, less negative social attributes, and were more likely to accept lifestyle recommendations than Japanese older adults. This study contributes to our understanding of older adults’ perceptions of robot voice and provides design implications for robots that make recommendations to older adults.

Finding its Voice: The Influence of Robot Voice on Fit, Social Attributes, and Willingness to Use Among Older Adults in the U.S. and Japan

Participatory robot design projects with older adults often use multiple sessions to encourage design feedback and active participation from users. Prior projects have, however, not analyzed the learning outcomes for older adults across co-design sessions and how they support constructive design feedback and meaningful participation. To bridge this gap, we examined the learning outcomes within a “longitudinal panel.” This panel comprised seven co-design sessions with 11 older adults of varying cognitive abilities over six months, aimed at designing a robot to guide a photograph-based conversational activity. Using Nelson and Stolterman’s framework of the hierarchy of design-learning, we demonstrate how older adult pan-elists achieved multiple design-learning outcomes — capacity, confidence, capability, competence, courage, and connection — which allowed them to provide actionable design suggestions. We provide guidelines for conducting longitudinal panels that can enhance user design-learning and participation in robot design.

"Give it Time:" Longitudinal Panels Scaffold Older Adults' Learning and Robot Co-Design

Bittersweet Snapshots of Life: Designing to Address Complex Emotions in a Reminiscence Interaction between Older Adults and a Robot

Robots could support older adults' well-being by engaging them in meaningful conversations, specifically to reflect on, support, and improve different aspects of their well-being. We implemented a system on a QT social robot to conduct short autonomous conversations with older adults, to help understand what brings them feelings of joy and meaning in life. We evaluated the system with written surveys and observations of 12 participants including older adults, caregivers, and dementia care staff. From this, we saw the need to improve user experience through personalized interaction that better support older adults as they talk about well-being. Improving the interactions will involve improving the conversation flow, detecting emotions and nonverbal cues, and natural language processing to extract topics around well-being.

https://dl.acm.org/doi/pdf/10.1145/3568294.3580067

Spill the Tea: When Robot Conversation Agents Support Well-being for Older Adults

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Hiroki Sato

Author Tools

Chat about Author

Papers

Finding its Voice: The Influence of Robot Voice on Fit, Social Attributes, and Willingness to Use Among Older Adults in the U.S. and Japan

"Give it Time:" Longitudinal Panels Scaffold Older Adults' Learning and Robot Co-Design

Bittersweet Snapshots of Life: Designing to Address Complex Emotions in a Reminiscence Interaction between Older Adults and a Robot

Spill the Tea: When Robot Conversation Agents Support Well-being for Older Adults