Journal article

Authors list: Martin, Paul P.; Graulich, Nicole

Publication year: 2023

Pages: 407-427

Journal: Chemistry education research and practice

Volume number: 24

Issue number: 2

ISSN: 1109-4028

eISSN: 1756-1108

DOI Link: https://doi.org/10.1039/d2rp00287f

Publisher: Royal Society of Chemistry (RSC)

Abstract: 
In chemistry, reasoning about the underlying mechanisms of observed phenomena lies at the core of scientific practices. The process of uncovering, analyzing, and interpreting mechanisms for explanations and predictions requires a specific kind of reasoning: mechanistic reasoning. Several frameworks have already been developed that capture the aspects of mechanistic reasoning to support its formative assessment. However, evaluating mechanistic reasoning in students' open responses is a time- and resource-intense, complex, and challenging task when performed by hand. Emerging technologies like machine learning (ML) can automate and advance the formative assessment of mechanistic reasoning. Due to its usefulness, ML has already been applied to assess mechanistic reasoning in several research projects. This review focuses on 20 studies dealing with ML in chemistry education research capturing mechanistic reasoning. We developed a six-category framework based on the evidence-centered design (ECD) approach to evaluate these studies in terms of pedagogical purpose, rubric design, construct assessment, validation approaches, prompt structure, and sample heterogeneity. Contemporary effective practices of ML-based formative assessment of mechanistic reasoning in chemistry education are emphasized to guide future projects by these practices and to overcome challenges. Ultimately, we conclude that ML has advanced replicating, automating, and scaling human scoring, while it has not yet transformed the quality of evidence drawn from formative assessments.

Harvard Citation style: Martin, P. and Graulich, N. (2023) When a machine detects student reasoning: a review of machine learning-based formative assessment of mechanistic reasoning, Chemistry education research and practice, 24(2), pp. 407-427. https://doi.org/10.1039/d2rp00287f

APA Citation style: Martin, P., & Graulich, N. (2023). When a machine detects student reasoning: a review of machine learning-based formative assessment of mechanistic reasoning. Chemistry education research and practice. 24(2), 407-427. https://doi.org/10.1039/d2rp00287f