Author(s):
Improving Conceptual Understanding in STEM
Using Web-Based Writing Applications
James Becker, Montana State University
Need: Conceptual-based writing exercises are a valuable tool to detect common misconceptions among students in gateway STEM courses and form a strong complement to standard computation-based problem solving. Effective feedback on student responses to such writing exercises is key to correcting misconceptions. Unfortunately, the time commitment required to administer and evaluate writing exercises limits their use.
This project explores the use of web-based writing applications that include immediate and personalized feedback enabled via Natural Language Processing (NLP) techniques.
Guiding Questions: There are three guiding questions governing the project: (1) What is the impact of a writing-centric approach in terms of enhancing students’ conceptual understanding, metacognitive skill, and ultimate success in a foundational course on electric circuit analysis? (2) Is there evidence the impact of the writing exercises depends upon the mode with which the exercises are administered, and the feedback provided? (3) To what extent is the web-based writing application template transferable to conceptual writing exercises across a variety of gateway STEM courses?
Outcomes: Students were divided into at-risk to fail and not at-risk to fail groups based upon their performance on an initial conceptual-based writing exercise that had previously been found to be a strong predictor of a student’s ultimate performance in a circuit analysis course. Potential growth in conceptual understanding of DC circuit concepts was measured using a pre-test / post-test assessment on scores from a validated multiple-choice exam as well as scores on the conceptual portion of the course’s final exam in which students were asked to justify their responses.
The control in this study are those students who received only the initial conceptual-based writing exercise (i.e. no intervention). There were two intervention groups. The first intervention group received a total of five conceptual-based writing exercises, including the initial used to classify students according to risk. All writing exercises were administered during class time in hardcopy form with instructor feedback given immediately after the quiz to the class as a whole. The second intervention group received the same five writing exercises, administered however as web-based applications with direct feedback given through the web application. Results have consistently shown that not-at risk students achieve higher scores on both the pre-test and post-test multiple-choice exam compared to at-risk students. With the intervention, however, at-risk students’ pre-test / post-test growth in performance is greater, with their post-test scores approaching those of the not at-risk class. T
Broader Impacts: The project is now at the point of working with external participants to develop conceptual-based writing exercises for a diverse set of STEM courses. To date, the implications of the project suggest that at-risk students in particular may benefit from including conceptual-based writing exercises in STEM courses as a supplement to standard computation-based problems.
Coauthors
Douglas Hacker, Madison, Wisconsin
