Advancing Measurement Technology in Educational Intervention Research to Study Individual, Contextual, and Implementation Heterogeneity
Open Access
- Author:
- Husmann, Kyle
- Graduate Program:
- Human Development and Family Studies
- Degree:
- Doctor of Philosophy
- Document Type:
- Dissertation
- Date of Defense:
- May 05, 2023
- Committee Members:
- Charles Geier, Professor in Charge/Director of Graduate Studies
Karen Bierman, Outside Unit Member
Timothy Brick, Co-Chair & Dissertation Advisor
James Diperna, Outside Field Member
Zita Oravecz, Co-Chair of Committee - Keywords:
- education
heterogeneity
technology
measurement
educational intervention - Abstract:
- For the past two decades, educational intervention research has rallied around the common goal of determining “what works in education” to inform education practice and policy (U.S. Department of Education, 2017). To study individual, contextual, and implementation processes and their interaction with educational interventions requires dramatically scaling up the amount of data we collect in educational research (Bryan et al., 2021). Existing technology has the potential to dramatically increase our data collection capacity without the cost and burden associated with traditional measurement approaches. In this dissertation I present two pilot studies that demonstrate tractable and immediately accessible ways that mobile technology can expand our ability to test theory about educational processes. In the first study, I present results from an Ecological Momentary Assessment (EMA) study that used a mobile app to collect parents' daily levels of parental self-efficacy for their child’s home literacy activities. Although cross-sectional studies have shown relationships between parental self-efficacy and their involvement with their child’s literacy activities in the home environment (e.g. Dulay et al., 2018; Tazouti & Jarlégan 2016; Giallo, 2013), the extent to which these relationships represent day-to-day relationships existing within parents is not known. I test competing predictions from social cognitive theory (Bandura, 2000) and resource allocation theory (Yeo & Neal, 2006) about the within-person dynamics of parental self-efficacy and their child’s home literacy activities. In the second study, I demonstrate how mobile technology can augment materials in an existing intervention in order to capture low-burden, intensive longitudinal measures of implementation and implementation-related processes. I present a proof-of-concept mobile app that I developed to replace the paper forms of an existing parent-implemented literacy intervention, and I share results from a pilot test of its feasibility and acceptability. I demonstrate the app’s ability to automatically collect dense longitudinal measures during the intervention with negligible burden to parents, children, and researchers. In analyses using the novel data I collected in the pilot, I test hypotheses about within-person relationships existing in parents’ and children’s emotional experiences during literacy activity sessions and their future engagement with the intervention. I conclude with a vision for the future of measurement and data collection in educational intervention research. I introduce Cattell’s (1952) developmental “Data Box” framework and show how it can organize the types of meaningful differences existing in evaluations of educational interventions. Finally, I argue that expanding our measurement capability to study heterogeneity in all its forms is not only necessary for advancing our scientific knowledge about education, but also a moral imperative to ensure that applications of our scientific knowledge provide equitable benefits to the world.