IGNORING HIERARCHICAL DATA STRUCTURE IN ITEM RESPONSE THEORY ANALYSES: IMPLICATIONS FOR EDUCATIONAL AND PSYCHOLOGICAL RESEARCH
Open Access
Author:
Nolan, Katherine A
Graduate Program:
Educational Psychology
Degree:
Doctor of Philosophy
Document Type:
Dissertation
Date of Defense:
September 07, 2016
Committee Members:
Pui-Wa Lei, Dissertation Advisor/Co-Advisor Pui-Wa Lei, Committee Chair/Co-Chair Hoi Suen, Committee Member Wayne Osgood, Committee Member Mosuk Chow, Outside Member
Keywords:
Nested Multilevel Modeling IRT Hierarchical
Abstract:
Educational research often involves hierarchical or nested data such as, students nested within schools or classrooms, which violates the independence assumption and may lead to incorrect standard error estimates and inflated Type I error rates when traditional analyses are used. The consequences of ignoring hierarchical or multilevel data structure in general linear models have been well examined and documented. However, little is known about the consequences of ignoring multilevel data structure on multilevel IRT models, which are frequently used to estimate student latent ability.
The purpose of this study is to determine the consequences of disregarding nesting on IRT analyses by systematically investigating different factors that might influence the estimation of IRT parameters. Two-level IRT data is simulated and subsequently analyzed with a single-level model ignoring clustering and an appropriate two-level model. Results showed that sample size, number of clusters, level of dependency (ICC), and number of items influenced the parameter recovery and estimation of single-level and two-level IRT parameters. Recommendations are made as to the use and necessity of multilevel IRT model.
