Author(s):
Need: Instructors and students benefit from feedback when it is timely, actionable, and accurate. Researchers can leverage longitudinal, fine-grained data collected at scale to develop learning trajectories within and across STEM disciplines and investigate the efficacy of different pedagogical practices. We are developing cognitive diagnostic formative assessments for the LASSO platform to support instructors, students, and researchers in individualizing instruction at scale. Cognitive diagnostics are adaptive tests that optimize the questions students are asked to accurately and precisely measure both a student’s overall ability in a domain and their mastery of skills that underlie that domain. Guiding Question: Our initial work is focused on developing a cognitive diagnostic for introductory mechanics that can measure the underlying skills that cut across content areas within mechanics. The research assessed the extent to which our model of five skills fit the data from six different research-based assessments for mechanics and mathematics. The five skills were applying vectors, conceptual relationships, algebraic manipulation, visualizations, and using calculus. The data came from 23,382 students who completed assessments on the LASSO platform (lassoeducation.org). Each RBA measured three to five of the skills. Outcomes: Using deterministic inputs, noisy ‘and’ gate (DINA) models, we found a good fit for these skills in the data. Assessments with five or more items for a skill showed high accuracy in identifying skill mastery. By combining these six instruments into a single item bank, the Mechanics Cognitive Diagnostic will assess students skills and overall ability across the content areas in introductory mechanics areas. We plan to expand this item bank to enhance the diagnostic’s precision and versatility, enabling its use for both comprehensive testing and targeted weekly assessments. Broader Impacts: Our development of clear, actionable reports for instructors aims to facilitate individualized instruction. Individualizing instruction helps to address hidden educational debts that society owes to minoritized groups. It ensures that the unique needs of every student are met, rather than teaching only to the average, and prevents neglecting the needs and abilities of students from groups excluded from STEM.
Coauthors
Ben Van Dusen and Vy Le, Iowa State University, Ames, IA; Xiuxiu Tang, Yuxiao Zhang, Amirreza Mehrabi, Jason W. Morphew, and Hua Hua Chang, Purdue University, West Lafayette, IN
