Manipulation checks

Abstract: Participants are not always as diligent in reading and following instructions as experimenters would like them to be. When participants fail to follow instructions, this increases noise and decreases the validity of their data. This paper presents and validates a new tool for detecting participants who are not following instructions – the Instructional manipulation check (IMC). We demonstrate how the inclusion of an IMC can increase statistical power and reliability of a dataset.

Abstract: This article addresses the conceptualization and definition of message variables in persuasion effects research. Two central claims are advanced. First, effectbased message variable definitions (in which a message variation is defined in terms of effects on psychological states, as when fear appeal variations are defined on the basis of differences in aroused fear) impede progress in understanding persuasion processes and effects and hence should be avoided in favor of definitions expressed in terms of intrinsic message features. Second, when message variations are defined in terms of intrinsic features, message manipulation checks, under that description, are unnecessary but similar measures may usefully be understood and analyzed as assessments of potential mediating states. One enduring question in communication research is how and why persuasive messages have the effects they do. But some important conceptual aspects of this subject seem to have suffered from inattention, with resulting needless confusion and impaired research progress. The particular focus of concern in this paper is the set of complex relationships among experimental message variations, message manipulation checks, persuasive outcomes, and mediating states. The purpose is to point to some systematically different sorts of research claims that arise in the context of studying persuasion effects, with an eye to clarifying the different burdens of proof—and corresponding data-analytic treatments—appropriate for each and thereby to untangling some of the complexities and confusions that have arisen in this research domain. Two central claims will emerge from this analysis: First, effect-based message variable definitions impede progress in understanding persuasion processes and effects and hence should be avoided in favor of definitions expressed in terms of intrinsic message features. Second, when message variations are defined in terms of intrinsic features, message

Abstract: Background: Health behavior theories converge on the hypothesis that attitudes, norms, and self-efficacy are important determinants of intentions and behavior. The present review analyzed whether changing attitudes, norms, or self-efficacy leads to changes in intentions and behavior in studies that used random assignment, manipulation checks, and post-intervention measures of outcomes. Methods: Literature searches obtained 193 experimental tests that met the inclusion criteria, which were meta-analyzed via STATA. Findings: Experimentally induced changes in attitudes, norms, and self-efficacy all led to medium-sized changes in intention (d+ = .50, .41, and .50, respectively), and engendered small to medium-sized changes in behavior (norms-d+ = .20; attitudes-d+ = .37; self-efficacy-d+ = .46). These effect sizes generally were not qualified by the moderator variables examined (e.g., study quality, methodological characteristics). Discussion: The present review (a) indicates that correlational studies (and related meta-analytic syntheses) overestimate the effect of cognitions on intentions and behavior, (b) lends novel, experimental support for key predictions from health behavior theories, and (c) demonstrates that interventions that modify attitudes, norms, and self-efficacy are effective in promoting health behavior change.