Development of a differential item functioning (DIF) procedure using the hierarchical generalized liner model: a comparison study with logistic regression

Open Access
Author:
Kim, Wonsuk
Graduate Program:
Educational Psychology
Degree:
Doctor of Philosophy
Document Type:
Dissertation
Date of Defense:
December 11, 2002
Committee Members:
  • Hoi Kin Suen, Committee Chair
  • Jaeyong Lee, Committee Member
  • Michael J Rovine, Committee Member
  • Rayne Audrey Sperling, Committee Member
  • Robert James Stevens, Committee Member
Keywords:
  • differential item functioning
  • hierarchical generalized linear model
Abstract:
Based on Kamata’s item analysis model (2001), an extension for differential item functioning procedure was developed and the applicability was examined. The Kamata’s item analysis model is a type of hierarchical generalized linear model for item analysis, in which items are considered as nested in examinees. Kamata showed that the item difficulty estimates and person ability estimates are mathematically equivalent to the Rasch model. Extending the Kamata model by adding matching variable and group membership variable and their interaction terms in the level-2 model, the approach to DIF, which is named the Hierarchical Generalized Linear Modeling DIF, was constructed. The main purpose of this study was to examine the equivalency of this procedure with a traditional DIF method, the logistic regression DIF method. An English test that consisted of 60 items with 4 sub-scales for freshmen’s diagnostic purpose was analyzed to answer the research questions in this study. For the extensive investigation of the two DIF methods, four different group membership variables were administrated; Male vs. Female, White vs. Asian, White vs. Black, Asian vs. Black. The comparisons were made with the coefficients for the group membership variable and interaction term. In addition to that, p-values of the regression coefficients were compared. The results of the empirical comparison show that the DIF outputs are highly similar to each other in both uniform DIF and non-uniform DIF. Beyond the similarity with the logistic regression DIF method, features of the HGLM DIF method and its strengths and weaknesses were discussed. Furthermore non-parametric examination of the DIF item by using the TestGraf program was considered in order to find the prominent difference in pattern of DIF identified by specific procedures