Cognitive Diagnostic Assessment in University Statistics Education: Valid and Reliable Skill Measurement for Actionable Feedback Using Learning Dashboards
TL;DR: In this paper , the validation process of constructing detailed diagnostic information on a set of skills, abilities, and cognitive processes (so-called attributes) from students' item response data with diagnostic classification models is described.
read more
Abstract: E-learning is increasingly used to support student learning in higher education, facilitating administration of online formative assessments. Although providing diagnostic, actionable feedback is generally more effective, in current practice, feedback is often given in the form of a simple proportion of correctly solved items. This study shows the validation process of constructing detailed diagnostic information on a set of skills, abilities, and cognitive processes (so-called attributes) from students’ item response data with diagnostic classification models. Attribute measurement in the domain of statistics education is validated based on both expert judgment and empirical student data from a think-aloud study and large-scale assessment administration. The constructed assessments provide a valid and reliable measurement of the attributes. Inferences that can be drawn from the results of these formative assessments are discussed and it is demonstrated how this information can be communicated to students via learning dashboards to allow them to make more effective learning choices.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Design and implementation of an AI‐enabled visual report tool as formative assessment to promote learning achievement and self‐regulated learning: An experimental study
Xiaofang Liao,Xuedi Zhang,Zhifeng Wang,Heng Luo +3 more
TL;DR: An AI‐enabled visual report tool comprising six modules is developed and implemented that generates data‐driven, user‐friendly reports and can not only enhance students' learning achievement and self‐regulated learning over time but also increase their test anxiety.
18
A Survey of Models for Cognitive Diagnosis: New Developments and Future Directions
Fei Wang,Weibo Gao,Qi Liu,Jiatong Li,Guanhao Zhao,Zheng Zhang,Zhenya Huang,Mengxiao Zhu,Shijin Wang,Wei Tong,Enhong Chen +10 more
- 07 Jul 2024
TL;DR: This paper surveys current cognitive diagnosis models, focusing on machine learning-based methods, and provides a comprehensive review of recent trends, model structures, and applications, while discussing future directions and releasing two Python libraries for easy access and implementation.
Properties and performance of the one-parameter log-linear cognitive diagnosis model
Lientje Maas,Matthew J. Madison,Matthieu J. S. Brinkhuis +2 more
TL;DR: The one-parameter log-linear cognitive diagnosis model (1-PLCDM) is a novel diagnostic classification model that exhibits desirable measurement properties and improved interpretability compared to general diagnostic classification models.
3
Improving Instructional Decision‐Making Using Diagnostic Classification Models
W. Jake Thompson,Amy K. Clark +1 more
TL;DR: Diagnostic classification models (DCMs) can be used to improve instructional decision-making by providing fine-grained reporting on skills that students have mastered.
1
References
13. students analyzing data: research of critical barriers
Clifford Konold,Alexander Pollatsek,Arnold D. Well,Allen Gagnon +3 more
- 01 Jan 1996
TL;DR: The authors argued that some of the difficulty people have in formulating and interpreting statistical arguments results from their not having adopted such a perspective, and that they make sense of statistics by interpreting them using more familiar, but inappropriate, comparison schemes.
28
Guiding learners through technology-based instruction: The effects of adaptive guidance design and individual differences on learning over time.
Adam M. Kanar,Bradford S. Bell +1 more
TL;DR: The authors investigated the interactive effects of guidance design (i.e., framing of guidance information) and individual differences (e.g., pretraining motivation and ability) on learning basic and strategic task skills over time.
A General Framework for the Validation of Embedded Formative Assessment
TL;DR: In this article, the authors applied the argument-based approach to validation to the context of formative assessment, which resulted in a proposed interpretation and use argument consisting of a score interpretation and a score use.
23
Validating Student Score Inferences with Person-Fit Statistic and Verbal Reports: A Person-Fit Study for Cognitive Diagnostic Assessment.
Ying Cui,Mary Roduta Roberts +1 more
TL;DR: In this paper, a two-stage procedure was used to evaluate person fit for a diagnostic test in the domain of statistical hypothesis testing, which helped to identify the misfits of item-score vectors to the cognitive model used in the design and analysis of the diagnostic test, and discover the reasons of misfits so that students' problem-solving strategies were better understood and their performances were interpreted in a more meaningful way.
14