Construct Validity in Software Engineering
Dag I. K. Sjøberg,Gunnar R. Bergersen +1 more
- 06 Mar 2021
TL;DR: A quantitative analysis of the awareness of construct validity in the software engineering literature between 2000 and 2019 and a qualitative review of 83 articles about human-centric experiments published in five high-quality journals between 2015 and 2019 suggest a minimalist approach to construct validity.
read more
Abstract: Empirical research aims to establish generalizable claims from data. Such claims involve concepts that often must be measured indirectly by using indicators. Construct validity is concerned with whether one can justifiably make claims at the conceptual level that are supported by results at the operational level. We report a quantitative analysis of the awareness of construct validity in the software engineering literature between 2000 and 2019 and a qualitative review of 83 articles about human-centric experiments published in five high-quality journals between 2015 and 2019. Over the two decades, the appearance in the literature of the term construct validity increased sevenfold. Some of the reviewed articles we reviewed employed various ways to ensure that the indicators span the concept in an unbiased manner. We also found articles that reuse formerly validated constructs. However, the articles disagree about how to define construct validity. Several interpret construct validity excessively by including threats to internal, external, or statistical conclusion validity. A few articles also include fundamental challenges of a study, such as cheating and misunderstandings of experiment material. The diversity of topics discussed makes us recommend a minimalist approach to construct validity. We propose seven guidelines to establish a common ground for addressing construct validity in software engineering.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Threats to validity in software engineering research: A critical reflection
Roberto Verdecchia,Emelie Engström,Patricia Lago,Per Runeson,Qunying Song +4 more
TL;DR: This position paper critiques the treatment of threats to validity in software engineering research, highlighting shortcomings such as inadequate categorization and afterthought consideration, and proposes improvements to enhance the validity and rigor of empirical studies.
34
40 Years of Designing Code Comprehension Experiments: A Systematic Mapping Study
TL;DR: A systematic mapping study of 95 source code comprehension experiments published between 1979 and 2019 provides a basis for subsequent discussion of the huge diversity of design options in the face of a lack of basic research on their consequences and comparability.
On researcher bias in Software Engineering experiments
Simone Romano,Davide Fucci,Giuseppe Scanniello,Maria Teresa Baldassarre,Burak Turhan,Burak Turhan,Natalia Juristo +6 more
TL;DR: In this paper, the authors present two studies to increase our body of knowledge on researcher bias in Software Engineering (SE) experiments, including: (i) QRPs potentially leading to researcher bias, (ii)causes behind researcher bias; and (iii) possible actions to counteract researcher bias with a focus on, but not limited to, blinding techniques.
Deep Learning Safety Concerns in Automated Driving Perception
Stephanie Abrecht,Alexander Hirsch,Shervin Raafatnia,Matthias Woehrle +3 more
TL;DR: This paper introduces an additional categorization for a better understanding as well as enabling cross-functional teams to jointly address the concerns of safety concerns, leveraging feedback from various domain and safety experts in the field.
Improving the Reporting of Threats to Construct Validity
Dag I. K. Sjøberg,Gunnar R. Bergersen +1 more
- 08 Jun 2023
TL;DR: In this paper , the authors identify, categorize, assess and quantify discussions of threats to construct validity in empirical software engineering literature and use the findings to suggest ways to improve the reporting of construct validity issues.
References
The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations.
Reuben M. Baron,David A. Kenny +1 more
TL;DR: This article seeks to make theorists and researchers aware of the importance of not using the terms moderator and mediator interchangeably by carefully elaborating the many ways in which moderators and mediators differ, and delineates the conceptual and strategic implications of making use of such distinctions with regard to a wide range of phenomena.
Perceived usefulness, perceived ease of use, and user acceptance of information technology
TL;DR: In this article, the authors developed and validated new scales for two specific variables, perceived usefulness and perceived ease of use, which are hypothesized to be fundamental determinants of user acceptance.
54.1K
Perceived Usefulness, Perceived Ease of Use, and User
Perceived Usefulness,Fred D. Davis +1 more
- 01 Jan 1989
TL;DR: Regression analyses suggest that perceived ease of use may actually be a causal antecdent to perceived usefulness, as opposed to a parallel, direct determinant of system usage.
47.9K
Case Study Research Design And Methods
Christine Nadel
- 01 Jan 2016
TL;DR: The case study research design and methods is universally compatible with any devices to read and is available in the book collection an online access to it is set as public so you can download it instantly.
23K
Convergent and discriminant validation by the multitrait-multimethod matrix.
TL;DR: This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components.
17.4K