Tracking and Improving Strategy Adaptivity in a Complex Task

TL;DR: An algorithm is developed that accurately identifies the strategies that people employ in a complex decision-making task based on task behavior and builds a strategy coaching module that enhances the task interface to support consistent application of a strategy and identification of suboptimal strategies.

Abstract: Strategies are a major component of increasing expertise and performance in complex tasks. High performers often have better strategies than low performers even with similar practice. Relatively little research has examined how people form and modify strategies in tasks that permit a large set of possible strategies. One challenge with such research is determining strategies based on behavior. We have developed an algorithm that accurately identifies the strategies that people employ in a complex decision-making task based on task behavior. In this paper, we report different methods to identify strategies that human participants are using in a complex decision-making task and document the efficacy of our methods. Participants have difficulty applying strategies consistently and thereby fail to gain useful performance-related feedback about the strategy’s effectiveness. A further challenge is to identify the optimal strategy to use as the properties of the task change over time. We refer to these challenges as strategy consistency and strategy adaptivity. These analyses have led to the construction of a strategy coaching module that enhances the task interface to support consistent application of a strategy and identification of suboptimal strategies. We report initial data on the effectiveness of this strategy coaching module.