Pre-trained large language models (“LLMs”) like GPT-3 can engage in fluent, multi-turn instruction-taking out-of-the-box, making them attractive materials for designing natural language interactions. Using natural language to steer LLM outputs (“prompting”) has emerged as an important design technique potentially accessible to non-AI-experts. Crafting effective prompts can be challenging, however, and prompt-based interactions are brittle. Here, we explore whether non-AI-experts can successfully engage in “end-user prompt engineering” using a design probe—a prototype LLM-based chatbot design tool supporting development and systematic evaluation of prompting strategies. Ultimately, our probe participants explored prompt designs opportunistically, not systematically, and struggled in ways echoing end-user programming systems and interactive machine learning systems. Expectations stemming from human-to-human instructional experiences, and a tendency to overgeneralize, were barriers to effective prompt design. These findings have implications for non-AI-expert-facing LLM-based tool design and for improving LLM-and-prompt literacy among programmers and the public, and present opportunities for further research.

/pdf/why-johnny-cant-prompt-how-non-ai-experts-try-and-fail-to-1ix7sl6j.pdf

Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts

Artificial Intelligence (AI) plays an increasingly important role in improving HCI and user experience. Yet many challenges persist in designing and innovating valuable human-AI interactions. For example, AI systems can make unpredictable errors, and these errors damage UX and even lead to undesired societal impact. However, HCI routinely grapples with complex technologies and mitigates their unintended consequences. What makes AI different? What makes human-AI interaction appear particularly difficult to design? This paper investigates these questions. We synthesize prior research, our own design and research experience, and our observations when teaching human-AI interaction. We identify two sources of AI's distinctive design challenges: 1) uncertainty surrounding AI's capabilities, 2) AI's output complexity, spanning from simple to adaptive complex. We identify four levels of AI systems. On each level, designers encounter a different subset of the design challenges. We demonstrate how these findings reveal new insights for designers, researchers, and design tool makers in productively addressing the challenges of human-AI interaction going forward.

https://dl.acm.org/doi/pdf/10.1145/3313831.3376301

Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design

The hybrid cellular automaton (HCA) algorithm is a methodology developed to simulate the process of structural adaptation in bones. This methodology incorporates a distributed control loop within a structure in which ideally localized sensor cells activate local processes of the formation and resorption of material. With a proper control strategy, this process drives the overall structure to an optimal configuration. The controllers developed in this investigation include two-position, proportional, integral and derivative strategies. The HCA algorithm combines elements of the cellular automaton (CA) paradigm with finite element analysis (FEA). This methodology has proved to be computationally efficient to solve topology optimization problems. The resulting optimal structures are free of numerical instabilities such as the checkerboarding effect. This investigation presents the main features of the HCA algorithm and the influence of different parameters applied during the iterative optimization process. DOI: 10.1115/1.2336251

Topology Optimization Using a Hybrid Cellular Automaton Method With Local Control Rules

The Wizard of Oz prototyping approach, widely used in human-computer interaction research, is particularly useful in exploring user interfaces for pervasive, ubiquitous, or mixed-reality systems that combine complex sensing and intelligent control logic. The vast design space for such nontraditional interfaces provides many possibilities for user interaction through one or more modalities and often requires challenging hardware and software implementations. The WOz method helps designers avoid getting locked into a particular design or working under an incorrect set of assumptions about user preferences, because it lets them explore and evaluate designs before investing the considerable development time needed to build a complete prototype.

http://dm.lcc.gatech.edu/~jlee0/jaemin/asset/pervasive.pdf

Wizard of Oz support throughout an iterative design process

Devices capable of gestural interaction through motion sensing are increasingly becoming available to consumers; however, motion gesture control has yet to appear outside of game consoles Interaction designers are frequently not expert in pattern recognition, which may be one reason for this lack of availability Another issue is how to effectively test gestures to ensure that they are not unintentionally activated by a user's normal movements during everyday usage We present MAGIC, a gesture design tool that addresses both of these issues, and detail the results of an evaluation

http://dmrussell.net/CHI2010/docs/p2159.pdf

MAGIC: a motion gesture design tool

Speech-based user interfaces are growing in popularity. Unfortunately, the technology expertise required to build speech UIs precludes many individuals from participating in the speech interface design process. Furthermore, the time and knowledge costs of building even simple speech systems make it difficult for designers to iteratively design speech UIs. SUEDE, the speech interface prototyping tool we describe in this paper, allows designers to rapidly create prompt/response speech interfaces. It offers an electronically supported Wizard of Oz (WOz) technique that captures test data, allowing designers to analyze the interface after testing. This informal tool enables speech user interface designers, even non-experts, to quickly create, test, and analyze speech user interface prototypes.

Suede: a Wizard of Oz prototyping tool for speech user interfaces

SUEDE is a speech interface prototyping tool that enables rapid, iterative creation of prompt-response speech interfaces. It explicitly supports iterative design, allowing a designer to quickly create an interface prototype, conduct user studies, and analyze the test data in a single tool. SUEDE offers an electronically supported Wizard of Oz (WOz) technique that captures user test data. Before building the tool that we present here, we built a paper and an interactive prototype and tested the prototypes with a total of fifteen users. We also interviewed six professional speech UI designers at their workplace about their current design practice. These designers encouraged us to build a tool that lets them start with examples when designing and simulate recognition error during testing. Our current tool is a culmination of this iterative design.

/pdf/suede-iterative-informal-prototyping-for-speech-interfaces-hpkt7w0d2y.pdf

SUEDE: iterative, informal prototyping for speech interfaces

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Jack Chen

Author Tools

Chat about Author

Papers

Suede: a Wizard of Oz prototyping tool for speech user interfaces

SUEDE: iterative, informal prototyping for speech interfaces