Model-based evaluation is using a model of how a human would
use a proposed system to obtain predicted usability measures
by calculation or simulation. These predictions can replace or
supplement empirical measurements obtained by user testing.
In addition, the content of the model itself conveys useful information about the relationship between the user’s task and
the system design.

/pdf/model-based-evaluation-epj4v36tpa.pdf

Model-based evaluation

Analyzing a task into Goals, Operators, Methods, and Selection rules (GOMS) is an established method for characterizing a user's procedural knowledge. When combined with additional theoretical mechanisms, the resulting GOMS model provides a way to quantitatively predict human learning and performance for an interface design, in addition to serving as a useful qualitative description of how the user will use a computer system to perform a task. This chapter focusses on GOMS models as a taskanalytic notation and how to construct them.

GOMS Models for Task Analysis

Task analysis is the process of understanding the user's task thoroughly enough to help design a computer system that will effectively support users in doing the task. By task is meant the user's job or work activities, what the user is attempting to accomplish. By analysis is meant a relatively systematic approach to understanding the user's task that goes beyond unaided intuitions or speculations, and attempts to document and describe exactly what the task involves. The design of functionality is a stage of the design of computer systems in which the useraccessible functions of the computer system are chosen and specified. The basic thesis of this article is that the successful design of functionality requires a task analysis early enough in the system design to enable the developers to create a system that effectively supports the user's task. Thus the proper goal of the design of functionality is to choose functions that are both useful in the user's task, and which together with a good user interface, results in a system that is usable, being easy to learn and easy to use.

/pdf/task-analysis-and-the-design-of-functionality-4lhjw2s38a.pdf

Task Analysis and the Design of Functionality

Cognitive engineering—a multidisciplinary field that focuses on improving the fit between humans and the systems they operate—emerged in the early 1980s and has many applications, including intelligence analysis and command and control. The APL Cognitive Engineering Program leverages the Laboratory’s strengths in both cognitive engineering research and human-system integration. The challenge is to implement those strengths through changes in APL’s culture, organization, engineering policy and practices, and a course of action toward implementing these changes.

cognitive engineering: understanding human interaction with complex systems

Human performance models are abstractions, usually mathematical or computational, that attempt to explain or predict human behavior in a particular domain or task. This includes a wide range of tec...

A History and Primer of Human Performance Modeling

This paper presents the lessons learned when a computational GOMS modeling tool was used to evaluate user interface concepts and team structure designs for a new class of military shipboard workstations. The lessons are both encouraging and cautionary: For example, computational GOMS models scaled well to a large and complex task involving teams of users. Interruptability and working memory constructs had to be added to conventional GOMS model concepts. However, two surprises emerged: First, the non-psychological aspects of the model construction were the practical bottleneck. Second, user testing data in this domain were difficult to collect and lacked definition, meaning that the model provided a better characterization of the design details than the user testing data. Included in these lessons are recommendations for future model applications and modeling methodology development.

/pdf/computational-goms-modeling-of-a-complex-team-task-lessons-1a4f9xax3w.pdf

Computational GOMS modeling of a complex team task: lessons learned

: Computational tools and techniques for modeling team performance have advanced significantly in recent years. However, there have been few efforts to combine complementary modeling approaches. In the Manning Affordability Initiative, we have applied three modeling technologies to experimental data from a single domain (air defense warfare), a single scenario, and common watchstation technologies (current AEGIS technology and an advanced prototype). The conclusion of this multi-year project in early 2002 offers an opportunity to review the findings. The proposed panelists will summarize a human-in-the-loop experiment conducted to provide modeling data and present findings from efforts to integrate three modeling approaches for design and design validation. Team Optimal Design (TOD) focuses on team modeling. The Integrated Performance Modeling Environment (IPME) uses a general task modeling technique that applies well to individuals or teams. The GOMS Language Evaluation and Analysis Tool (GLEAN) combines individual models of users interacting as a team.

http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA461076&Location=U2&doc=GetTRDoc.pdf

Complementary Methods of Modeling Team Performance

: This report summarizes an investigation of the application of a spatialized audio interface over headphones for use in submarine sonar operations and training that was performed in the period from December 2007 through September 2010 as part of a larger ONR study led by the Johns Hopkins University Applied Research Laboratory with engineering support from the Naval Undersea Warfare Center, Newport. NSMRL, with collaborators at University of Michigan and New York University, provided applied binaural research on human auditory processing of spatialized (or 3D) audio over headphones. A comprehensive summary is presented of a series of experiments in six focus areas quantifying human listener performance as a function of variations in critical parameters of this new audio interface. Recomendations are also made for further study in four of the focus areas where significant performance improvement may be gained.

/pdf/studies-on-a-spatialized-audio-interface-for-sonar-3ajxwuk2v2.pdf

Studies on a Spatialized Audio Interface for Sonar

While computational team modeling techniques and tools have advanced significantly, relatively few efforts have incorporated several distinct approaches toward a common goal. The Office of Naval Research sponsored Manning Affordability Initiative has applied 3 modeling technologies to a single domain (air defense warfare), a common scenario, and watchstation technologies (current AEGIS technology and human-centered design prototypes). This has provided an unusual opportunity to investigate these tools, and ways in which they can be used together. Team Optimal Design (TOD) focuses on team modeling. Integrated Performance Modelling Environment (IPME) uses a general task modeling technique that applies well to individuals or teams. The GOMS Language Evaluation and Analysis Tool (GLEAN) combines individual models of users interacting as a team. This panel discussion focuses on these 3 methods and their combined uses. Further, a human-in-the-loop experiment conducted to provide data against which to validate the computational models will be summarized.

http://pro.sagepub.com/content/45/24/1723.full.pdf

Panel on Complementary Methods of Modeling Team Performance

: Small teams of human performance models perform tasks in a simulated Air Defense Warfare (ADW) scenario to compare two designs for an advanced multi-modal watch station (MMWS). In the original design, the models select and perform tasks based on their own knowledge of the on-going air warfare situation as acquired through visual search of individual tactical situation (TACSIT) displays and information windows and verbal communications among model operators. In a revised design, an intelligent agent Task Manager (TM) monitors the situation and generates a list of tasks on a separate display for the operator to monitor. While exercises on a prototype TM with human teams were confounded by user issues, the modeling study produced a quantitative comparison of task execution latency and total critical tasks between the two designs that showed generally improved performance for the revised design, thereby demonstrating the utility of human performance modeling for the evaluation of systems for complex team tasks.

A Comparison of Air Defense Warfare Task Performance with and without an Automated Task Manager Using a GOMS Modeling Tool

Popular Tasks used by Researchers

How can I help with your research?

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

We encountered an error while loading the panel. Please try refreshing the page.

Something went wrong

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

Anyone with the link can view this chat. You can switch it back to private anytime.

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Thomas P. Santoro

Author Tools

Chat about Author