The importance of validity testing – Goal Orientation and Goal Affordance

Kelsie Bernot
Associate Professor of Biology
North Carolina A&T State University

NeedWe proposed to combine two high-impact practices – undergraduate research and service learning – in the context of service learning, course-based undergraduate research experiences (SL-CURE) and explore how this hybrid course design influences students’ perceptions of, interest in, and motivation to pursue research careers. This combination of service learning and research may be particularly powerful, because women, historically underserved groups, and first-generation students often experience role incongruity, in which they place high value on communal goals – working with and helping others, yet they see research as a more agentic pursuit that may not fulfill communal goals (Eagly and Diekman, 2004; Diekman 2010) We implemented and assessed SL-CUREs at three diverse institutions: (1) a large, public, historically black university; (2) a small, private, predominantly white liberal arts college; (3) a large, public Hispanic serving institution. Guiding Questions:This presentation will focus on one of our three guiding research questions:RQ1: Does incorporating service learning into CUREs (SL-CURES) along with interventions highlighting how research can support communal goals effectively challenge student misconceptions about scientific research and increase their interest in research? I will discuss the validity testing that we did on one Goal Affordance and Goal Orientation instruments (Diekman 2010) to assess the congruency between students’ personal goals and whether they perceive research as being capable of fulfilling those goals.Outcomes Prior research on the Goal Affordance and Goal Orientation instruments supports a 2- (Diekman, 2010) or 5- factor model (Knekta, 2019). Confirmatory Factor Analysis (CFA) on a comparative pilot sample of our students (n=188) did not support prior research on either model. After attempting Exploratory Factor Analysis, neither eigenvalue nor scree analysis supported a clear model to pursue. We then piloted a novel instrument in interviews / focus groups in which students sorted the items from the prior survey into a Venn Diagram depending on whether that goal was only personally important to them, only afforded by research, was both personally important and afforded by research, or neither personally important nor afforded by research. We performed semi-structured interviews / focus groups with the novel instrument, with inductive and theoretical coding of the transcripts. We will discuss the findings of the interviews / focus groups in assessing the novel instrument that suggest potential reasons for our lack of CFA confirmation.ContributionOur early data confirms the importance of extensive validity testing and reveals insights into how different populations may interpret and value items differently based on lived experiences. However, we will also demonstrate our ability to pivot both our analysis plan as well as the course implementation as COVID required online instruction for our first iteration of the course. This project will directly impact the diversity of the STEM workforce by enriching the educational experiences of historically underserved students. This project will directly impact the diversity of the STEM workforce by enriching the educational experiences of historically underserved students.


Mike Wilton, UCSB, Santa Barbara, CA; Lalo Gonzalez, UCSB, Santa Barbara, CA; Michele Malotky, Guilford College, Greensboro, NC; Erika Wiliams, NC A&T SU, Greensboro, NC; Breanne Bygrave, NC A&T SU, Greensboro, NC; Allana Matthews, NC A&T SU, Greensboro, NC