Addressing Systemic Issues in STEM Education: Potential and Perils of XR Technologies

Caption: Authors Michele McColgan, Ph.D. and Jason Morphew, Ph.D. provide an overview of the MARVLS app, an augmented reality app, which helps students develop spatial reasoning skills and conceptual understanding of STEM concepts. Developing these skill sets can increase student persistence in STEM. (Video credit: Jason Morphew.)

To address 21^st century challenges, a large, well-trained, and most importantly, a diverse STEM workforce is needed. By the year 2025, over three million jobs requiring a degree in STEM will be unfilled in the United States.¹ There is a need to increase the recruitment and retention of diverse students across all STEM fields.² Educational systems must effectively prepare students to apply STEM learning to solve authentic problems. These systems must be equitable and culturally situated to benefit all learners.^3,4 While strides have been made in both areas, just 25% of undergraduate students enroll as STEM majors, and only half of these students graduate with a STEM degree.⁵ Many students discontinue their pursuit of a STEM degree within the first two years after negative experiences in introductory courses. These courses tend to use direct instruction and focus on developing fluency with formalisms^6-8 and can display inequitable outcomes for underrepresented students,^9-13 with these courses often acting as STEM gatekeepers.^14-16

One of the factors contributing to the current landscape of STEM post-secondary education is deficit- vs growth-mindset framing. Deficit framing focuses on what those who do not persist in STEM lack. In contrast, growth-mindset framing assumes all students can learn and focus on culturally embedded and relevant instruction. The design and implementation of extended reality (XR) technologies including virtual reality (VR), augmented reality (AR), and mixed reality (MR) have the potential to either minimize or exacerbate the impact of framing.

How can we design XR learning environments so that students can apply intuitive and well-developed spatial reasoning evident in everyday contexts to novel, complex, and abstract STEM contexts? We draw on theories of embodied cognition to understand how thinking is connected to movement and principles of embodied learning to improve equity by applying these theories to the design of XR environments.

Spatial Reasoning is a Key Skill

Learning within STEM entails developing conceptual understandings and theoretical models involving concepts with complex interactions between abstract and unseeable objects, forces, and constructs. Electricity and Magnetism (E&M) is an introductory course requiring strong spatial reasoning. E&M is required for most science, engineering, computer science, and pre-medicine majors. E&M students use spatial reasoning to develop models visualizing interactions between charged particles, electric and magnetic fields, and other unseen or abstract topics. These models are built upon in other courses as students progress through their majors.

Many students struggle with spatial reasoning, particularly female students, and students from lower socioeconomic status backgrounds.^17-20 Because spatial reasoning is correlated with performance and persistence in STEM,^21-24 it acts as a barrier for success in STEM²⁵. However, these same students demonstrate spatial reasoning in everyday settings by successfully navigating their environments. The problem is not a student’s lack of spatial reasoning ability, but rather learning environments that don’t facilitate transfer of existing skills.

Embodied Learning Facilitates Engagement, Learning, and Transfer

Abstract conceptual understanding is grounded in embodied experiences and representations. Theories of embodied cognition view cognitive processes of thinking and learning as grounded in bodily actions and systems of perception, as well as brain functioning. The precise mechanisms of embodied thought are still debated; all theories agree that thinking and reasoning are distributed across cognitive and perceptual-motor systems such that conceptual reasoning originates in bodily interaction and is internalized as simulated action.^26-30 Conceptual understanding impacts attention, how we interact with our environment, and how we ascribe meaning.^31,32 Similarly, movement within social and physical environments impacts conceptual understanding and the application of conceptual models.^33,34

Derived from theories of embodied cognition, embodied learning activities facilitate meaningful connections between body movements and conceptual understanding.³⁵ A key feature is that instruction is grounded in students’ own physical interactions, either prompted or spontaneous, while explicit focus is directed to connections between actions and conceptual understanding. Within embodied learning, actions frequently enact metaphors between intuitive spatial reasoning and key principles in STEM.³⁶ Because XR technologies overlay digital representations of unseen objects and abstract conceptions onto personally relevant environments, XR has the potential to extend embodied experiences and make productive conceptual metaphors salient and manipulable.

Enhance Equity and Impact in XR Using Design Principles

XR can advance student learning and equity in STEM, but this potential is not inherent in the XR platform. For example, VR can provide immersive environments with detailed visual representations. However, if the learner is merely clicking a button to move objects, conceptual learning will be limited. For XR to impact STEM education, it is imperative that designers consider the mapping of student actions to conceptual understanding. Several design principles aligned with embodied cognition have been described by various researchers:

1. Digital visualizations of unseeable objects or abstract conceptions should be manipulable using embodied actions like movement or gesture.^37-39
2. Embodied actions within XR environments should use body-based metaphors aligned with conceptual understanding to map actions to digital visualizations.^38-40
3. Embodied actions should be scaffolded and recur across contexts.^{37, 41}
4. XR environments should provide opportunities for collaborative interactions within the physical and the digital environment.^{38, 41}
5. XR environments must provide opportunities for explicit reflection on actions, how actions relate to visualizations and formalisms.^{40, 42}

Adopting these principles can improve equity.

Design for Equity

XR learning environments must be designed and implemented to be accessible and useable by all learners. While the costs for devices needed to access XR learning environments have decreased, public funding for education across K-16 has declined.^{43, 44} The impact of budget cuts tends to impact schools from low socioeconomic status neighborhoods the most. To design XR STEM learning environments without consideration for equity of access serves to maintain or increase existing performance gaps.

Design of XR systems for diverse users must be a conscious decision. The presence of inequity in interactions across technology advances are widespread and well documented, such as bias across gender and skin tone in facial recognition systems^45-47 and optical sensors,⁴⁸ racial bias in AR environments,⁴⁹ gender bias in VR,^{50, 51} and bias in machine learning and artificial intelligence algorithms and training sets.^52,53 These outcomes are likely not examples of planned inequity. However, because equity issues in technology are well known, failing to design for diverse users represents a conscious and purposeful design decision.

Our work with ELASTIC³S^{33, 54} is an example of designing for equity in interactions. ELASTIC³S incorporates immersive AR science simulations to teach cross-cutting concepts in science. Embodied actions are linked to conceptual reasoning, through the Microsoft Kinect V2 sensor which detects students’ movements and infers a skeletal mapping of those movements in real time. Extensive testing with diverse users demonstrated that inequitable outcomes were possible based on the lighting of the learning environment; therefore, the environment was designed to reduce back lighting. We also addressed diversity in mobility by implementing a Hierarchical Hidden-Markov Model (HHMM) to achieve “one shot” gesture recognition where the user defines the action and trains the system.⁵⁵ The use of HHMM allows individuals with mobility issues to interact with AR systems by performing actions consistent with their capabilities.

What does this look like in practice?

The MARVLS App is an augmented reality app for students to explore models of abstract or 3D concepts in STEM. Topics are drawn from an introductory E&M course. This course requires students to form 3D mental models for unseen objects and abstract concepts, connect 2D representations to the 3D models they represent, and relate equations and formalisms to both representations. Instructors traditionally use demonstrations, videos, or simulations to help students, however, many students still struggle. MARVLS uses a free app and a paper cube (equity of access) to digitally place virtual models depicting unseen objects and abstract concepts (Principle 1).

Figure 1: 2D image connected to 3D model for magnetic field of a current-carrying wire. (Left) 3D model of a magnetic field and current carrying wire. The 2D representation is found on the visualization and in a grey box. (Right) Students interact with the 2D image in grey box by pressing it. This augments the visual representation by adding a green arrow to highlight the magnetic field at the 2D plane. (Image credit: McColgan)

Students manipulate the virtual models by holding the cube and rotating it to explore changes in the virtual model. This scaffolding of action allows students to draw on their embodied spatial orientation abilities to translate between 2D and 3D representations and connect their conceptual models to formalisms such as equations or graphs (Principle 2).

Students using MARVLS to explore abstract conceptual models drawing on their inherent understanding of spatial orientation. (Image credit: McColgan)

MARVLS facilitates collaboration by designing activities that allow students working in teams to develop conceptual models by exploring E&M concepts (Principle 3). In these activities students explore the interaction of virtual models as they interact with each other in physical space.

Figure 3: (Left) Students working in teams using MARVLS. (Right) Collaborative MARVLS exploring magnetic flux in a copper ring (left cube) as the result of a magnetic field (right cube). (Image credit: McColgan)

Figure 4: Student reflecting on the connections between their interactions with MARVLS and the abstractions and formalisms related to the content while still engaged in the interactions. (Image credit: McColgan)

MARVLS provides opportunities for reflection on interactions with the physical cube and digital model (Principle 4). Students manipulate aspects of the digital models by modifying current flow, shape of components, and orientation of the system to explore complex interactions between objects and fields. Activities then scaffold students to map these interactions to formalisms, such as Lorentz forces, Gauss’s Law, and Ampere’s Law, as well as abstractions, such as mathematical equations like Maxwell’s equations. By connecting equations and formalisms to a variety of actions grounded in conceptual metaphors and intuitive understanding, MARVLS facilitates conceptual learning while preserving equity in interactions.

XR Design for Maximum Impact

XR platforms provide an opportunity to ground STEM instruction in students’ inherent spatial reasoning by activating their visuospatial resources, in place of direct instruction based in learning formalism and symbolic manipulation. Systemically scaffolding instruction from embodied cognition perspectives can help students struggling with visuospatial reasoning. To ensure equity, designers of XR learning environments are encouraged to heed the following advice:

1. Choose the XR environment by considering the mapping of bodily action to conceptual understanding.
2. Use productive body-based metaphors for actions that impact unseen objects or abstract constructs.
3. Purposefully and explicitly design for equity in XR environments.
4. Incorporate the physical and social environments in which students exist.

The affordances XR provides are not inherent in the platform. With purposeful design and implementation in STEM courses, XR learning environments have the potential to advance learning and equity in STEM.

Further Engagement

How to Bring MARVLS to Your Classroom

The MARVLS apps are available for free in the Apple Store and in Google Play:

Android

iOS

Download a free merge cube.

Learn More about Embodied Cognition, Embodied Learning, and Embodied Design

• Engineering Education Podcast with Dr. Jason Morphew
• Readings that provide an introduction to embodied cognition and embodied learning:
  • Grounded Cognition – L. W. Barsalou²⁶
  • Making “Concreteness Fading” More Concrete as a Theory of Instruction for Promoting Transfer – E. R. Fyfe & M. J. Nathan²⁷
  • Embodiment as a Unifying Perspective for Psychology – A. M. Glenberg²⁸
  • An Embodied Theory of Transfer of Mathematical Learning – M. J. Nathan & M. W. Alibali²⁹
  • Six Views of Embodied Cognition – M. Wilson³⁰
  • What does it mean to say the mind is embodied – Art Glenberg
  • Educating the Embodied Mind Blog Post
  • Foundations of Embodied Learning – M. Nathan
• Readings that provide an introduction to XR Design Principles
  • Teaching Students to Think Spatially through Embodied Actions: Design Principles for Learning Environments in Science, Technology, Engineering, and Mathematics – D. DeSutter & M. Stieff³⁷
  • Emboldened by Embodiment: Six Precepts for Research on Embodied Learning and Mixed Reality – R. Lindgren & M. Johnson-Glenberg³⁸
  • Considerations for the Design of Gesture-Augmented Learning Environments – R. C. Wallon & R. Lindgren³⁹
  • The Necessary Nine: Design Principles for Embodied VR and Active Stem Education – M. C. Johnson-Glenberg⁴⁰
  • Designs for Grounded and Embodied Mathematical Learning – M. Nathan, C. Walkington, & M. I. Swart⁴¹
  • Embodiment and Embodied Design – D. Abrahamson & R. Lindgren⁴²
  • Learning with AR and MR – R. Lindgren
  • Human – Computer Interaction Seminar – V. Hart & E. Eastmond
  • Fireside chat – M. Johnson-Glenberg

Learn More About i²Engineering and ELATIC³S

• Learn more about i²Engineering
• Learn more about ELATIC³S

Acknowledgements

The authors acknowledge support from the National Science Foundation, awards DUE #2120446, and an instructional equipment grant, all which support research, practice, and institutional learning on developing, implementing, and promoting learning theory-based educational technology that promote equitable and inclusive STEM education for faculty and students. The authors also acknowledge the essential contributions of the many colleagues who have collaborated on the development of the embodied learning technology presented in this blog post. (1) MARVLS: Natalie Stagnitti, Justin Marotta, Ben McColgan, and Rebecca Lindell. (2) Siena College Physics course: George Hassel and Kamyar Pashayi. (3) ELATIC³S: Robb Lindgren, Nitasha Mathayas, James Planey, Jina Kang, Jose Mestre, Michael Junokas, Guy Garnett, Nicholas Linares, Greg Kohlburn, Sahil Kumar, and Ben Lane.