AAAS-IUSE
Summer Labs

Jacquelyn Chini, University of Central Florida; Erin Scanlon, University of Connecticut, Avery Point

Contact: jchini@ucf.edu

Description: This project has developed an observation protocol for Universal Design for Learning (UDL) in postsecondary STEM. UDL is a framework for proactively designing instruction to support variations in learners’ needs, abilities, and interests. An existing protocol, the UDL Observation Measurement Tool (UDL-OMT), was used, but it did not fully meet project needs. For example, there was not one-to-one alignment between the UDL framework (which has 31 checkpoints arranged into nine guidelines and three principles), and the UDL-OMT, and the protocol used holistic scoring across the observation. Since project goals were to communicate with postsecondary instructors about their instructional practice, the UDL-OMT items were expanded to include the observed practices of the instructors. In the current version of the revised protocol, the Universal Design for Learning Instructional Practice Observation Instrument (UDL-IPOP), each UDL checkpoint has been phrased as an observation item with exemplar practices based on prior work. Observers will use the tool to document the practices observed during intervals of time.

Desired Impact or Implications: A project participant described that many postsecondary STEM instructors design inaccessible classes not because they WANT to be inaccessible, but rather because they do not know what accessible and inclusive practices look like. This project component aims to aid instructors in incorporating accessible and inclusive practices in their instructional design as well as offer a tool to provide feedback to instructors as they implement these changes. Future plans include examining whether courses that use more UDL-aligned strategies have improved student outcomes and which practices have the highest impact.

Requested input: Some difficult decisions remain, such as the amount of time to use the interval-based protocol (e.g., 2 minutes, 5 minutes, 10 minutes) as well as some UDL checkpoints for which there were no or few practices. It would be useful to get feedback about how to select and test the optimal time interval as well as to brainstorm additional practices to include in the protocol.

Jenny Dauer, University of Nebraska; Amanda Sorensen, Michigan State University; P. Citlally Jimenez, University of Nebraska-Lincoln

Contact: pcjimenez@huskers.unl.edu

Description: Project staff seek review and feedback on their proposed teaching approach. A manuscript (in press) is below that describes the approach, centered on supporting students’ science literacy skills around decision-making and information literacy.

Desired Impact or Implications: The goal is to define “science literacy skills” and how they are developed in formal classrooms.

Requested input: In addition to general feedback about the perceived value and effectiveness of the teaching approach, feedback on how to adapt elements of this approach to disciplinary classrooms and ideas about potential student outcomes for further study would be particularly useful.

Perry Samson, University of Michigan-Ann Arbor

Contact: samson@umich.edu

Description: This project was put on hold due to COVID-19 restrictions; it requires measurements made in in-person courses. Now that restrictions have eased, project staff will use the survey instruments to gather student background data and changes in attitudes over the length of the course during the Fall 2021 semester.

Desired Impact or Implications: Data will be collected about students’ sense of belonging, their comfort with participating in in-class inquiry and activities. These data will be matched with the level of each student’s participation in in-class inquiry and activities.

Requested input: Project research was delayed, so the surveys are still untested. Project staff seek advice about how to test and/or improve these instruments before initiating measurements in the fall.

Review Project Deliverable: https://docs.google.com/forms/d/1Db8ZyrLgW4XAohvF2JLEvG9XqnbCV9KiLL8Rqqmx518/viewform?edit_requested=true

Eric Brenner, Pace University; Yao Wang, New York University; Jan Plaas, New York University

Contact: ebrenner@pace.edu

Description: Over four-week period students learn the importance of plants and then are asked to design an experiment on plant shoot movement, specifically gravitropism (movement against gravity). This entails having them learn how to cultivate plants and then run a falsifiable experiment on plant movement using their cell phones to record these movements thereby creating their own time-lapse movies. The IUSE supported cellphone App, Plant Tracer, will then be used to quantify various parameters in Arabidopsis thaliana (genetic model plant) movement.  The experiment tests what genes may be involved in plant shoot movement through the analysis of a discreet list of mutant plants in genes that have been implicated (in the literature) in root movement. Additionally, students are engaged in several learning activities to support the objectives of this project.

Desired Impact or Implications: The desired observed deliverable will be a significant increase in interest in plants as delineated via our pre- and post-test assessment. The goal of this project is to increase awareness, interest, and appreciation in plants and plant related careers. Increased interest in plants can be measured through changed attitudes towards plants, increased interest in plant facts, plant related hobbies (indoor and outdoor cultivation), increased pursuit in courses involving plant biology and increased pursuit of plant related careers including research, agriculture, plant-derived products and ecosystems management.

Requested input: The project team is looking to improve the curricular design to achieve their goals of improving plant interest. An outside review would help tailor the lab learning structure and lecture component on plant biology to more effectively improve student enthusiasm and interest. Besides the Plant Tracer plant movement experiment, students perform two additional plant experimental lab activities and two lecture assignments, which includes a home activity, to increase appreciation and knowledge of plants and plant careers.  An outside perspective would be very helpful to make the project design even more affective.

Review Project Deliverables:

Shelbi Kuhlmann; Jeffrey Greene; Matthew Bernacki; Abigail Panter; Kathleen Gates; Kelly Hogan, University of North Carolina at Chapel Hill

Contact: skuhlmann@unc.edu

Description: A targeted early intervention provided to undergraduate biology students in Spring 21 at UNC-CH is designed to support their learning and study strategies before they take the first exam in an introductory undergraduate biology course. The intervention leverages well-designed instructional videos to teach students about cognitive and self-regulated learning strategies that adhere to the science of learning. Within the intervention, students watch videos, answer questions about what they learned, and then reflect on their learning by writing about what they know well and what they still need to learn. The goal of this intervention is to help early STEM majors engage in productive learning strategies to increase their success in current and future STEM coursework.

Desired Impact or Implications: It is a challenge to effectively support student success in STEM. In the proposed work, after using data-driven models to identify STEM students most in need of support in active learning classrooms, digital support methods are tested, such as the multimedia intervention attached here, to help STEM students succeed.

Requested input: After several semesters of deploying various digital interventions, with increasing levels of required interactivity and time spent on task from students, a significant effect of these interventions was not evident. Feedback about how this intervention could be more generative and engage students in more instances of active learning that might lead to a benefit of this intervention would be appreciated as well as about how the design and/or content of the videos could be improved to better help students learn and use strategies that adhere to the science of learning. Any additional feedback about successfully intervening with undergraduate STEM students using digital interventions, such as this, would be appreciated.

Review Instructions: It is important to note that this intervention takes about an hour and a half to complete in its entirety (watching all videos and completing all questions). Please feel free to adjust your review of this product accordingly (e.g., watching only a few videos fully, and not responding, or typing N/A to questions). You will also need to enter the dummy email: onyen@unc.edu to progress past the first page of the intervention.

Ben Van Dusen, Iowa State University; Jayson Nissen, Nissen Research and Design

Contact: bvd@iastate.edu

Description: LASSO is a free online platform for hosting, administering, scoring, and analyzing research-based assessments across the STEM disciplines (Van Dusen, 2018). To use LASSO, instructors create a course, describe that course, and populate it with students and assessments they want administered. The students receive links to the pretests and posttests via email based on timelines set by the instructor. Students using LASSO first consent to have their data anonymized and included in LASSO’s database. Students then answer questions about their demographics and class activities before beginning the assessment. At any point the instructor can download all their students’ answers. After all data collection completes, the instructor can download a summary report with analysis of their student data or a file with all student responses.

LASSO currently hosts 46 instruments and has been used by 65,000+ students in 1,600+ courses from 70 institutions. The platform has seen continued year-over-year growth in users. Using LASSO provides similar student participation and performance as in-class administrations (Nissen, et al., 2018). Researchers from eight different institutions have used LASSO data in 17 peer-reviewed publications on student learning.

Desired Impact or Implications: LASSO is designed to provide instructors with the means to collected empirical evidence on the impacts of their teaching on STEM student learning, identity, and self-efficacy. This supports instructors in making evidence-based decisions on their pedagogical practices. For instructors that regularly administer pre-post research-based assessments in class, it frees up a significant amount of class time to be used for instruction.

Requested input: Feedback on how to improve LASSO is welcome, but staff also hope to reach new users who may not have heard about, or haven’t gotten around to, trying to use LASSO. In particular, education researchers who could use data from LASSO to power their research.

Review Project Deliverables:

Zhongzhou Chen, University of Central Florida

Contact: Zhongzhou.Chen@ucf.edu

Description: The project has created 70 online learning modules for introductory physics based on a mastery-learning instructional design. A core focus of the project is to power instructors to continuously improve those modules or design new ones based on detailed analysis of student interaction data. The presentation will include both the online learning modules and the latest data analysis and visualization tool under development. The goal is to enable instructors to use those data tools to “sense” students’ learning behavior, and dynamically evolve the course to meet diverse student learning needs.

Desired Impact or Implications: The mastery-based online learning modules allow students to actively engage with and adjust their learning according to the course content, whereas the analysis and visualization tools will allow the instructor and the course to dynamically adjust and continuously evolve to meet the needs of the students. Rather than relying predominantly on correct answer percentage, the new tools will be able to synthesize and visualize students’ self-regulating behavior to reveal previously hidden information about students’ SRL process.

Requested input: Both the module design and the visualization/analysis scheme under review can benefit from expert/peer feedback, especially since the visualization is predominantly targeted towards instructors. In addition, project staff seeks to determine if there are good ways to migrate the online learning modules to popular learning management systems and strategies to enhance impact of the results.

David Roundy, Oregon State University; Corinne A. Manogue, Oregon State University; Elizabeth Gire, Oregon State University, Tevian Dray, Oregon State University

 Contact: roundyd@physics.oregonstate.edu

Description: The Paradigms in Physics project developed a second-generation curriculum dissemination website that was intended to share materials (primarily student activities) developed over the past 20 years and that were housed on an earlier website. The new website actively serves curricular content to students in the classes of project staff. The external view of the site has not yet been honed.

Desired Impact or Implications: The project provides active engagement curricular materials for faculty teaching primarily middle-division and upper-division courses for Physics majors. The goal is to improve teaching within the physics major by sharing these materials.

Requested input: Feedback is needed on how visitors might use the new website.  Questions include:

• How do you search for active engagement curricular materials?
• What information would you need to decide whether to use a given activity? Is this information readily available?
• What information would you need to actually use a given activity in your class? Is this information readily available?

Frank Wang, LaGuardia Community College; Esther Isabelle Wilder, Lehman College, The City University of New York

Contact: fwang@lagcc.cuny.edu; Esther.Wilder@lehman.cuny.edu

Description: Feedback is requested for the instructional videos that are being developed (and will continue to be developed) for the faculty development program that is key to this grant. These videos are designed to align with the learning goals of this project and will cover a variety of topics such as calculating and interpreting percentages, making a bivariate/pivot table in Google sheets, preparing tables and graphs, and so on and so forth. The PI is in the process of developing and refining these instructional videos.

Desired Impact or Implications: The videos for the Data Analysis Research Experience (DARE) will complement an in-person/virtual training program and serve as a resource to both faculty teaching and students who are enrolled in DARE classes. Faculty participants in the DARE program are expected to make the videos a key component of their instruction and they will serve as a resource for students who will be working with spreadsheets to engage in hands-on data analysis. This project deliverables have been designed to serve faculty and students at Minority-Serving Institutions (MSIs) and especially Hispanic Serving Institutions (HSIs).

Requested input: Feedback is requested on how the videos can be strengthened, what would be helpful to include, what could be omitted, etc.

Tracy Hammond, Texas A&M University, College Station; Linsey, Weaver, Ray, Li, Krishnamurthy, Jaison, Merzdorf, Teo, Chen, Viswanathan, White

Contact: hammond@tamu.edu

Description: This project deliverable consists of an AI-based intelligent tutoring system named Sketchtivity that aids students in learning freehand 2D-perspective sketching skills by providing human-like feedback. Sketching is a critical skill that will help STEM students in idea generation, product concept formulation, externalizing mental models of systems, quick technical communication, and spatial visualization skills. Currently there is a significant lack of instruction in sketching in engineering classrooms. Due to the transition from hand drafting to CAD, most engineering faculty do not have the ability to teach and provide appropriate feedback. In addition, large engineering classrooms make it difficult for instructors to give students personalized feedback. Sketctivity makes it possible to teach sketching to engineering students with its comprehensive set of online lessons and exercises and universally accessible personalized feedback. Lessons will include step-by-step instruction that mimic those taught in professional paper drawing classes. The lessons start with the basics (lines, arcs, circles) and slowly progress towards more complex concepts and shapes (perspective drawing and 3D citystreet-scapes). The real-time visual feedback of their drawing skills relating to accuracy, speed, and fluidity is provided to the users. Sketctivity will be used by 2,500 undergraduate students across 3 different schools and more than 10 majors through direct or optional classroom use.

Desired Impact or Implications: Prior results indicate that Sketctivity improves visual communication skills and idea generation skills among STEM students. The results of the study will also help us to compare the benefits of human assessment and AI produced assessment. Spatial visualization skills have been repeatedly demonstrated as a critical skill for success in engineering discipline and have also shown to be lacking in minority and female students. Sketctivity is also expected to improve spatial visualization skills. We also envision helping minority students and females who struggle academically through this intervention and thereby improving retention. Further, Sketchtivity addresses the needs of an increasingly diverse workforce to teach students better freehand drawing skills beyond CAD-enabled experiences.

Requested input: The project team is seeking to introduce this AI- based tutoring system to faculty who would be interested in using it in their own classrooms. Broader feedback is requested from engineering educators regarding this tutoring intervention to teach sketching. How can this system be improved to further support equity and inclusion?

Review Project Deliverable:

Kent Crippen, University of Florida; Maria Korolev, University of Florida

Contact: kcrippen@coe.ufl.edu

Description: Participants are provided one of eight Design Challenges that were created as part of a two-semester curriculum sequence that is embodied in two new courses at the University of Florida, titled General Chemistry Laboratory for Engineers I & II. Design Challenge 1: Provide Access to Clean Water is a three-week Challenge where each phase, which also correspond to weeks in the course, is completed as a three-hour session in a teaching laboratory. This is the first full Challenge that students complete as part of General Chemistry Laboratory for Engineers I. This is offered as an interactive set of Canvas modules as well as a more traditional PDF document.

Desired Impact or Implications: This approach allows freshmen to experience the work of a professional engineer in a developmentally appropriate form as a means of learning the domain of chemistry. The laboratory environment offers unique opportunities for students to practice doing science and engineering and form links between macroscopic phenomena and molecular-level interpretations. Special consideration has been given to designing for populations sensitive to cultural and institutional issues, which include using universal/global engineering issues in lieu of engineering problems and formalized collaboration in lieu of competition. For engineering majors, contextualizing the learning of chemistry in such a way strengthens the connection between the domain knowledge of chemistry and its career application, which enhances interest, efficacy, and professional identity development. These claims are all supported by the continuing research efforts of this project team.

Requested input: This project team is interested in promoting dissemination of these materials as modules or complete courses and seek potential external collaborators for a potential level III IUSE proposal to investigate the process of scaling these activities across multiple contexts.

Review Instructions: This course has open enrollment. Participants can self-enroll via the URL or alternatively, they can sign up at: https://canvas.instructure.com/register Using the following join code: AW4MFW

Brian Skromme, Arizona State University

Contact: skromme@asu.edu

 Description: The project team requests feedback on a Circuit Tutor software system that provides unlimited randomly generated (both topology and element values) circuit problems along with fully explained example solutions and a step-based interface that accepts and immediately evaluates every step of a student’s work (including re-drawing circuit diagrams, entering equations using a template-based system, sketching waveforms, etc). Progressive levels of difficulty are provided.

Desired Impact or Implications: The goal is to improve student learning, motivation, and satisfaction in linear circuit courses and to maximize student success rates. The project team also hopes to improve conceptual learning in such courses for a better fundamental understanding. The general approach could be more broadly applied in other subject domains involving analysis of similar problems, though specific software would be needed in each topic area.

 Requested input: The project team would like input on the following:

1. design decisions to help prioritize development of different aspects of the system,
2. strategic advice on dissemination, publicity, and marketing strategies, and
3. advice on making the system broadly attractive to other institutions and instructors, to maximize the impact of the project, as well as on maximizing its pedagogical effectiveness would be useful.

Questions:

1. If you were considering adoption of these materials for your circuit course, what factors might limit your interest in doing so? (to help us maximize dissemination)
2. What types of improvements or changes do you think might most help improve student learning and retention in this system?

Review Instructions:

• Go to Circuit Tutor and log-in using the following credentials: circuittutor.com (Email: tveague.aaas@gmail.org; Password: LabTest) Review the following lessons:
  DC Single Node-Pair / Single Loop

  Bode Plots

  Videos demonstrating individual games

Elizabeth Johnson, James Madison University; Juhong Christie Liu, James Madison University

Contact: Juhong Christie Liu, liujc@jmu.edu

Description: This project has created five open educational resources (OER) online learning modules for laboratory methods and scientific inquiry skills. The learning unit in each module is structured with a guided learning inquiry model, modified from Process-Oriented Guided Inquiry Learning (POGIL), with unit title, learning objectives, prior knowledge, key concept exposition with guided inquiries, and concept check. The modular format is intended to provide flexibility for instructors to use as few or as many units as needed for a specific course, with blended learning experience through online learning management system (LMS) or in labs. These OER can also be accessed prior to or during field studies for student research. Lumen Learning, an OER education company, and Virginia Academic Libraries Consortium (VIVA) have supported the OER publication with two different platforms between 2016 and 2019 and then 2019-present respectively. In the duration, more interactive components for guided inquiries with H5P and embedded illustrative media have been gradually added to the project deliverables. Student learning was measured with a science writing rubric, LMS embedded quizzes and related homework. Accessibility was gradually enhanced as well in the project YouTube channel videos and other media.

Desired Impact or Implications: The project primarily plans to fill the gap of lacking appropriate instructional materials at the undergraduate level for laboratory techniques. The interactive OER publication allows removing significant barrier to accessing costly learning materials and limited and expensive lab equipment. The modified guided inquiries with knowledge exposition and skill demonstration are based on evidenced-based pedagogical practices in STEM education. With these readily-made and reusable ORE, instructors will be able to create active learning and research experience for student to access skills and knowledge necessary to succeed in the geosciences and other STEM fields.

Requested Input:

• How are these OER content practically and readily for geoscience lab methods class instruction, learning activities, and field studies?
• How would you use the interactive content in your classes? How can we modify the built-in learning assessment methods, rubric for assessing lab reports, and H5P learning objects?
• How can we modify these OER to make them sustainably reach broader audience?
• What strategies do you recommend for disseminating or expanding the use of these materials?

Review Instructions: Please review the Analytical Methods in Geosciences (AmiGEO) resources below. To access deliverable AmiGEO Rubric for Assessing Lab Reports, use passcode: AmiGEO2018

Julie Sexton, University of Colorado Boulder; Molly Jameson, University of Northern Colorado; Dina London, University of Northern Colorado; Jennifer Wenner, University of Wisconsin Oshkosh

Contact: julie.sexton@colorado.edu

Description: This project developed six affective domain lessons aimed at college students in introductory geology labs. In the lessons, students learn how math anxiety, interest, self-efficacy, stereotype threat, metacognition, and goal setting affect their learning and success related to math and quantitative problems in lab. The lessons provide students with strategies to improve their learning. Each lesson includes online self-paced reading, practice problems, low-stakes formative assessment, and application activity. The lessons can be completed fully online as they are currently structured. They can also be completed with a flipped teaching approach so that the reading, practice problems, and low-stakes formative assessment are completed online the week before a lab as homework. The application activity is completed during the first 10 minutes of lab to reinforce the online work lab. The six lessons can be completed in any order and can be dispersed across the semester to fit into an instructor’s schedule.

Desired Impact or Implications: There has been limited research to identify barriers for women to persist in geology. This project addresses the limited research by examining math as a barrier to female student persistence and success in geology.

Requested input: The project is scaling-up to implement the lessons in a larger number of classes. The project team would value input in the following areas before the scale-up:

1. What aspects are of the lessons are unclear and in need of modification?
2. What aspects of the lesson appearance and navigation in the online format need improvement?
3. The lessons are hosted on a temporary website. What sites or platforms could permanently host the lessons?
4. The formative assessments and application activities are in a Qualtrics survey so that the project team can collect student responses for a research study on the impact of the lessons. How can the assessments and application activities be provided to faculty while also collecting student responses for further study of lesson impact?
5. The lessons were developed with a focus on improving the affective domain for female students but could potentially benefit other marginalized groups. What aspects of the lessons need modification to be more inclusive of other marginalized groups?
6. What other recommendations do you have for improving the lessons?

Project review instructions: The link we provided in this submission takes you to the main page for the lessons. The six lessons are linked from that main page. When you select a lesson, it takes you to the lesson reading. At the end of the reading for the lesson, there is a link to the practice problems. After you complete the practice problems, there is a link to the formative assessment and application activity.

Review Project Deliverables: Please start with the following lessons for review:

Anna Stepanova, Texas A&M University; Christina Belanger, Texas A&M University; Tracy Hammond, Texas A&M University, Williford Blake, Texas A&M University; Christine Stanley, Texas A&M University; Sarah Raven, Texas A&M University

Contact: a.yu.stepanova@gmail.com

Description: FossilSketch was developed to teach undergraduate students the basics of micropaleontology in undergraduate geoscience courses and to provide a unique environment for students to practice and receive individual real-time feedback as they progress in learning about microfossils. FossilSketch focuses on two groups: benthic foraminifera and ostracoda. The software introduces students to micropaleontology and guides them through educational videos to learn why we study microfossils, their applications in geosciences, Foraminifera and Ostracoda research, their characteristics, and the principles of identification for each group. The novelty of the FossilSketch tool is in providing feedback on identification skills that previously have required more hands-on, qualitative face-to-face instruction as well as microscope use. FossilSketch has several components/sections that include:

• Introductory educational videos;
• Section to practice identifying various morphological features of microfossils using mini games;
• Section where students practice and master microfossil identification to genus level
• Section that introduces some of the applications of microfossils in research where students can practice identifying microfossils and reconstructing paleoenvironments.

Desired Impact or Implications: The project team envisions that FossilSketch will allow students to better understand the benefits of studying micropaleontology as well as improve their skills in identifying microfossils. The goal of this software is that it will improve recruitment and broaden participation in geoscience and other related STEM fields.  

Requested input: Our research team will benefit from exchanging, sharing and comparing ideas, as well as identifying collaborators to help broaden the use of this software across multiple universities.

Review Information: Here the project team submits a demonstration of the 3 educational videos and a presentation showing microfossil identification steps for Ostracoda and Foraminifera. Please review the following materials.

Rebecca Barnes, Emily Fischer, Melissa Burt, Ilana Pollack, Julie Maertens, Colorado State University; Sandra Clinton, University of North Carolina – Charlotte; Paul Hernandez, Texas A&M University; Mica Estrada, University of California – San Francisco

Contact: rbarnes@coloradocollege.edu

Description: Many scientists, regardless of career stage, state that their career advancement was due in part to effective mentoring and the role of mentors in their professional development. While some aspects of mentoring are still best served by formal mentoring relationships (i.e., a PhD committee advising the direction of a student’s research or an academic advisor during an undergraduate degree), a mentoring paradigm that empowers the mentee to actively expand their network and find the support needed—from different individuals, in different contexts, over different time spans—utilizing both informal and formal mentoring approaches, is usually the best approach to meet individual needs. This project deliverable discusses the different types of support we all need and how to build your own mentoring network.

Desired Impact or Implications: Reframing the idea of mentorship as a network of relationships that are beneficial to the mentee & mentor provides undergraduates with scaffold to think about what support they need, who they receive support from, and how to be intentional about creating a supportive mentor network for yourself with a collection of individuals (or programs and institutions); thus increasing their personal and professional resilience.

Requested input: The mentor mapping activity is not new to this program, but the project team has just started testing different online approaches given the realities of COVID-19 and to increase our program’s sustainability and scalability. This project deliverable is testing components of online delivery – with both synchronous (Facilitated Online Workshop using the Jamboard & Slide deck) and asynchronous versions (Online narrated video + Jamboard). The project team is seeking feedback on each of these elements.

• Online Video – The full mentoring exercise video is in the process of being created. Here is a link to the two-minute video portion that follows the participants filling in their own ecosystem map
• Jamboard – Each participant has two slides, which is linked to in the slide deck
• Slide Deck for facilitation

Matthew Bernacki, University of North Carolina at Chapel Hill; Erin Windsor, College of Southern Nevada;

Nancy Webb, College of Southern Nevada; Christopher Massa, College of Southern Nevada; Monty Evans, University of North Carolina; Sam Scudere-Weiss, Splunk, Inc.; Jon Hilpert, University of Nevada Las Vegas

Contact: Matthew.bernacki@gmail.com

Description: The linked documentation is meant to serve as a resource for colleges and universities that do not have an established campus unit that can provide the data management services necessary to provide the level of expertise, effort, and infrastructure necessary to develop, host, and maintain a data model necessary to observe students’ digital learning behaviors and enable educators to use them to inform instruction.

The documentation is meant to provide users with a high level overview of the way that they can:

• Establish a data stream from the Canvas learning management system using Live Events that capture student use of assets on a course site (i.e., “assets accessed”),
• Collect only course-relevant Live Events using a low cost solution on Amazon Web Services (i.e., an S3 bucket),
• Send these event data to an endpoint, Splunk, which can afford data modeling and machine learning analysis opportunities using a free to modestly priced local installation.

Desired Impact or Implications: Predictive models can provide accurate, early projections of students’ likelihood of success. These models require a good deal of historical data to develop a model and actively maintain the data model so that a prediction can be reapplied to future students. When students are predicted to be more likely to perform poorly than well, sending resources to students before a first major assessment improves their performance in the course. This documentation enables a university to build the infrastructure needed to develop such a system, so educators can use it to gain insight from data and provide timely support to learners.

Requested input: This documentation is designed to lower barriers to the development and provision of learning analytics solutions. Project documentation needs to be accessible to those who seek it, comprehensible to those who consider implementing a solution, and comprehensive so that the team of collaborators who can be assembled at a local campus has all the information they need to implement the solution with limited to no technical support.

The project team is asking reviewers to evaluate the accessibility and coherence of the documentation, the technical details provided, and embedded links to the additional documentation.

Philip Johnson, University of Hawaii

Contact: johnson@hawaii.edu

Description: STEM disciplines, such as computer science, must develop new and better ways to improve:

• engagement (i.e. create wider interest in pursuing CS),
• retention (i.e. create mechanisms to improve the chance that students, once pursuing a CS undergraduate degree, will complete it),
• and diversity (i.e. create ways to improve engagement and retention for women and underrepresented minorities).

To address these issues, this award developed the RadGrad project. The fundamental goal of the RadGrad Project is to provide students, faculty, and advisors with an alternative perspective on the undergraduate degree program, which traditionally boils down to a single kind of activity (coursework) and a single metric for success (grade point average).

Desired Impact or Implications: The goals of this project include improving degree planning, providing useful guidance, improving student engagement, improving student retention, improving undergraduate participation in research, improving student diversity, improving post-graduation professional success, and improving the degree experience for other disciplines. 

Requested input: This project would benefit from a review of the second generation RadGrad system. The project personnel are seeking suggested connections to researchers with similar interests.

Review Instructions: Please review the following pages of the website. The Motivation, Goals, and Basic Constructs pages provide an overview of the system. Feel free to review any other pages in the site that appear useful.

There is also a student introduction video & demo available. All demo instance accounts use the same password: “foo”.

Wendy Adams, Colorado School of Mines; Drew Isola, American Association of Physics Teachers

Contact: wkadams@mines.edu

Description: The goals for this project were to create six (four so far with two more coming out later this summer) engaging, brief videos for prospective teachers and faculty/staff to help them learn more about the profession and to further develop their interest in the profession. Each of the four videos were created to meet a need articulated by faculty who use the GFO resources to recruit teacher candidates.

Desired Impact or Implications: Specifically the project team aimed to create videos that were:

• Two to four minutes in length;
• Effective in a range of applications including:
  1. in a course;
  2. as part of a longer presentation;
  3. on a department/program website;
  4. during a conversation with a student or colleague; and
  5. on the Get the Facts Out website and YouTube Channel for easy public access.

Requested input: The project team would like input on the perceived value of these resources. Would the IUSE audience find themselves using these videos and would they like to see more resources of this nature?

Andrew Ross, Eastern Michigan University. Additional co-authors: https://modules2.com/our-team/

Contact: aross15@emich.edu

Description: The project team has written course materials that provide opportunities for prospective middle- and high-school teachers to develop mathematical knowledge as it relates specifically to the work of teaching geometry, statistics, algebra, and modeling. These modules can be used in university mathematics courses by faculty at universities and colleges of all types across the United States. Each of the 4 topics has 3 modules, with each module taking about 1/3rd of a semester. Since it would not be feasible to get all 12 modules (4 semesters’ worth) reviewed, we are supplying only a sample from each of the 4 topics. Please contact us if more is desired.

Desired Impact or Implications: The project team hopes that these materials improve the education of pre-service middle- and high-school math teachers, by:

1. improving the learning of mathematics and statistics by teacher candidates in a way that is aligned with their needs, and
2. demonstrating how a class can be run according to equitable teaching practices (active learning, discussion-based, etc.)

 Requested input: This team is hoping to receive input on:

1. These materials were written by different teams in different styles:
   • The Statistics materials follow the “Workbook Statistics” format of having something to read, then a question, with space to write an answer. They would likely be working in groups, with the instructor doing whole-class discussions every now and then.
   • The Geometry materials tell the instructor what to do: gather the class’s answers on the homework, get them into groups, (I) lead a discussion, (II) give a handout & they work on it, (III) whole-group discussion, (IV) small groups jigsaw task, (V) new handout, etc.
   • The project team is somewhat concerned that a faculty member who teaches from one of the sets of materials in one semester might have a hard time transitioning to the other format if they teach another topic the next semester. The team is looking for ways to smooth any transition, without entirely reformatting any of the materials sets. Any thoughts or suggestions on easy ways to make the materials more unified (without having to be uniform) would be appreciated!
2. Do you have any ideas for advertising and getting faculty (both those with a math-education background, and “plain” math/stats faculty) to adopt the materials?
3. Do you have ideas for publishing the materials? This team really wants to be able to post them online for free as Open Educational Resources. But it would probably also help dissemination and adoption if they could be published by a respected publisher.

 Review Project Deliverable: The links below provide a sample of the topics.

Omar Hernández-Rodríguez, University of Puerto Rico; Wanda Villafañe-Cepeda, University of Puerto Rico; Gloriana Gonzalez, University of Illinois

Contact: omar.hernandez4@upr.edu

Description: This work is part of the project Developing Technological Pedagogical Content Knowledge of Pre-service Math Teachers by Enhancement of a Methods Course Using Instrumental Orchestration and Lesson Study Strategies. The main goal is to addresses the gap in teacher education research around connections between methods courses and clinical experiences. This project team redesigned an early methods course to include clinical practices to help pre-service secondary mathematics teachers (PSTs) learn how to lead classroom discussions with interconnectivity technology. In the methods course, the PSTs learned to create math lessons using the Teacher Desmos Activity Builder (TDAB). In their field placements, they planned lessons using TDAB with their mentor teachers. The rubric supports PSTs’ lesson planning with their mentor teachers by listing moves for using interactive technology and characteristics of the moves.

Desired Impact or Implications: PSTs will develop their technological pedagogical content knowledge (TPACK).  PSTs will learn how to design technology-based lessons that support students’ development of mathematical proficiency using interconnectivity technology that allows students to share their work and communicate mathematical ideas through multiple representations.

Requested input: There is a tendency to use predesigned tools to teach mathematics. The project team would like input on the viability of PSTs using development tools such as TBDA to design their own lessons.

Shandy Hauk, San Francisco State University; JenqJong Tsay, University of Texas Rio Grande Valley; Billy Jackson, University of Louisville

Contact: hauk@sfsu.edu

Description: Canvas an online learning site for college mathematics faculty professional mini-course on teaching future K-8 teachers.

Desired Impact or Implications: The professional learning materials for novice college mathematics instructors supports learning about equitable college teaching. Project research indicates that such instruction improves undergraduate learning.

Requested input: The course site is being updated based on feedback from the field test (last year). The project would benefit from some “fresh eyes,” gathering feedback on what is clear and what still needs to be better explained on the website.

Review Instructions: If you have issues enrolling on the website, please contact Shandy Hauk (hauk@sfsu.edu): https://canvas.instructure.com/courses/2735443

Hollylynne Lee, NC State University; Stephanie Casey, Eastern Michigan University; Rick Hudson, University of Southern Indiana; Gemma Mojica, NC State University

Contact: hollylynne@ncsu.edu

Description: The Enhancing Statistics Teacher Education with E-Modules [ESTEEM] project is committed to changing the data and statistics learning experiences in grades 6-12 classrooms by engaging future teachers in meaningful teacher education materials. To reduce barriers to access, our materials are provided for free, use tools easily accessible in classrooms (e.g., CODAP), and is packaged in e-modules for import into learning management systems (LMS). The ESTEEM project has designed 3 modules and 2 large summative assessments to be used with preservice middle and high school mathematics teachers in online environments. Many instructors have adapted the materials for use in face-to-face or hybrid courses, or used them with other teacher audiences; they are distributed with a Creative Commons license.

Desired Impact or Implications: ESTEEM intended to:

• produce statistics teacher education curriculum materials,
• prepare mathematics teacher education (MTE) faculty to effectively use materials,
• successfully prepare mathematics teachers to engage future students in meaningful statistics lessons.

There are 300+ registered users in the portal. The project conducted workshops and webinars with 69 MTEs. Research efforts involved MTE field testers in 2018-2020.

In the next phase, this project hopes to scale-up and improve materials to reach a broader audience through a stronger focus on data science, attention to teaching statistics and data in other content areas, and teaching statistics at the elementary level.

Requested input: The project team is requesting input on:

• Strategies for expanding use of materials beyond secondary mathematics teacher preparation. What is needed to make materials applicable for elementary mathematics teacher preparation? How can the materials be expanded to incorporate data science education and data for social justice with connections to other disciplines (science, social studies)? What are strategies for growing a network of ESTEEM users and professional learning opportunities for faculty?
• Strategies for sustaining availability and usability of e-modules. Current approach is to have all materials in ESTEEM portal so faculty can peruse them, and have LMS export files containing all ESTEEM materials that can be imported into a course in 3 different LMSs (Moodle, Blackboard, Canvas), plus a Common Cartridge for importing modules into any LMS. This is a time-intensive process for making changes to materials and pushing out revisions to faculty. What could be done to make the e-Modules easiest to use by faculty while balancing the workload of sustaining them? The team is seeking advice on disseminating materials in more efficient ways that still provides flexibility for instructors to modify content to meet course needs.

 Review Information: To review the materials, please go to http://go.ncsu.edu/esteem and create a free account. You will then be able to review the online modules directly in the ESTEEM portal. On the right hand side of the portal, you will find links to download the materials in three LMSs along with directions and support videos.

Lynnette Michaluk, West Virginia University; Miller, P., West Virginia University; Stewart, G., West Virginia University; Koenig, K., University of Cincinnati; Henderson, R., Michigan State University

Contact: Lynnette.michaluk@mail.wvu.edu

Description: Much of the efforts to specifically support STEM students’ skills and knowledge of experimental design in undergraduate science content courses predates the Next Generation Science Standards (NGSS), and thus assessment instruments are not well correlated with the NGSS practices. The goal of this project was to develop and validate assessment items for an instrument that deconstructs the NGSS practice of Planning and Carrying Out Investigations (PCOI) into relevant sub-practices. Prior research and NGSS policy documents clearly describe these sub-practices at each grade band level: elementary, middle, and high school (NGSS Lead States, 2013; National Research Council, 2011). The elementary and middle school grade sub-practices provided the basis for the assessment instrument, as it was initially developed to assess pre-service K-8 teachers’ skills, knowledge, and pedagogical disciplinary knowledge of experimental design. Because early results were promising, project staff have expanded the application of this assessment for use with undergraduate students in other STEM courses, as the intention is to create a concept inventory useful for assessing PCOI in both preservice teachers and undergraduates in other STEM courses.

Desired Impact or Implications: The goal is to provide an assessment instrument that is highly correlated with the NGSS practice PCOI.

Requested input: The project seeks to make the instrument more useful for STEM teachers and their students.

Autar Kaw, University of South Florida; Mayank Pandey, University of South Florida

Contact: kaw@usf.edu

Description: This project aims to develop interactive simulations to improve student learning in a Numerical Methods course.  These simulations are being created using the Phet Simulation framework as it allows a robust and reliable way to present the simulations irrespective of the platform, device, or browser. Testing and review of these simulations will help to further improve their quality, including the future ones. Eight simulations have been developed and about 16 more would be formed in the Fall semester.

Desired Impact or Implications: The simulations will impact the learning and teaching of Numerical Methods courses nationwide. Many times, students do not follow the algorithm, and simulations are another modality to do so. It is desirable for students to develop simulations themselves but doing so for the whole course would not be tractable. The project team is also writing documentation so that others can use the framework to develop simulations for their STEM courses.

Requested input: Testing and review would improve the quality of the current and future simulations. The project is developing lessons and questions to accompany the simulations to determine best practices in the use of simulations beyond general manipulation. How can this project deliverable best serve the community so that others can adopt the developed practices and save the initial time of development?

Corinne Manogue, Oregon State University; Tevian Dray, Oregon State University; David Roundy, Oregon State University; Elizabeth Gire, Oregon State University

Contact: corinne@physics.oregonstate.edu

Description: Information about a freely-available, online math methods textbook, building on a successful online textbook covering multivariable calculus and electrostatics will be shared. The new text greatly extends the coverage of these previous texts, while also incorporating new state-of-the art technology to present interactive mathematics.

Desired Impact or Implications: The Paradigms in Physics project at Oregon State University, begun in 1996 and constantly evolving ever since, represents a holistic redesign of the entire upper-division curriculum for physics majors. Key features include attention to what students know, pedagogical strategies that are aligned with how people learn, and just-in-time development of mathematical skills and knowledge.

This project deliverables math methods text is intended to support the mathematical content of this curriculum *as used in physics*, emphasizing geometric reasoning and the use of multiple representations. The goal is to provide an intuitive interface across multiple platforms, while serving both as a standalone resource for physics majors and as a resource for instructors in both mathematics and physics. Further design goals include providing multiple tracks to allow exploration at several levels, and the use of high-quality, interactive graphics.

Requested input: This deliverables chosen format is PreTeXt, a markup language designed to seamlessly produce both print and online versions from the same source, although the materials are primarily designed for online, interactive use. PreTeXt is designed to facilitate interactive use, with such features as collapsible sections, high-quality mathematics typesetting and cross-referencing.  Perhaps most importantly, PreTeXt also allows easy embedding of interactive SageMath and GeoGebra sessions, providing the ability to include such features as rotatable, 3-dimensional graphs, and graphical examples whose parameters can be controlled by sliders.

The optimal form of active engagement materials varies, depending on how the book is being used. Worked examples might be best for use as a reference, guided exploration might be best for independent study, and open-ended questions might be best for use as a small-group activity during class. For the purposes of the IUSE Project Office Hours, the project team drafted alternate versions of one particular activity and would like advice on which one(s) to include. Can a single version address all three uses?

Review Project Deliverable: The project team is requesting review of three specific pages of their online textbook:

Brian Winkel, SIMIODE

Contact: BrianWinkel@simiode.org

Description: SIMIODE is a 501(c)3 nonprofit that received funding through this NSF IUSE proposal to support faculty and students in motivating, teaching, and learning differential equations using modeling throughout. SIMIODE has developed three sustainability efforts with the advice and consulting from NSF-funded Science Gateways Community Institute. These are

• SCUDEM, a student challenge in which teams submit videos for judging internationally,
• EXPO, an international conference that engages colleagues from around the world who are interested in modeling to motivate teaching and learning in differential equations coursework, and
• SIMIODE Digital textbook. All available at simiode.org. The project team would like to discuss these and other possible sustainability activities with respect to methods, Open Educational Resource (OER) materials, and building community.

Desired Impact or Implications: All three efforts (and all future sustainability efforts) are in support of faculty to teach differential equations, a pivotal STEM course, using models throughout the course to motivate student learning. SIMIODE wishes to increase engagement in the three initiatives described above and therefore enhance fiscal support for SIMIODE programs and platform.

Requested input: SIMIODE seeks assistance in its efforts to promote these activities while maintaining the true spirit of OER in the SIMIODE Community of Practice. Further, SIMIODE seeks new ideas for sustainability outreach, and avenues for engagement and community building. The project team seeks input on the following:

1. SCUDEM: Please provide feedback on the overall concept and practical considerations and suggestions for improvement in any aspect of SCUDEM (e.g., making the event fully accessible, fairness and inclusivity, technical details),
2. EXPO: The team seeks input on reasonableness of the concept of such a focused conference, the logistics and activities offered, and the inclusiveness of what is offered in terms of costs, the program usefulness to individuals, and how to best collect feedback from event participants.
3. Input on the SIMIODE Digital Textbook

Review Project Deliverables:

Additional Resources: Further, there are two groups in SIMIODE devoted to support students and faculty:

• Student: https://www.simiode.org/groups/simiode_textbook_student
• Teacher: https://www.simiode.org/groups/simiode_textbook_teacher