%0 Report %D 2006 %T Kernel-smoothed DIF detection procedure for computerized adaptive tests (Computerized testing report 00-08) %A Nandakumar, R. %A Banks, J. C. %A Roussos, L. A. %I Law School Admission Council %C Newton, PA %G eng %0 Journal Article %J Journal of Educational and Behavioral Statistics %D 2004 %T Evaluation of the CATSIB DIF procedure in a pretest setting %A Nandakumar, R. %A Roussos, L. A. %K computerized adaptive tests %K differential item functioning %X A new procedure, CATSIB, for assessing differential item functioning (DIF) on computerized adaptive tests (CATs) is proposed. CATSIB, a modified SIBTEST procedure, matches test takers on estimated ability and controls for impact-induced Type I error inflation by employing a CAT version of the SIBTEST "regression correction." The performance of CATSIB in terms of detection of DIF in pretest items was evaluated in a simulation study. Simulated test takers were adoptively administered 25 operational items from a pool of 1,000 and were linearly administered 16 pretest items that were evaluated for DIF. Sample size varied from 250 to 500 in each group. Simulated impact levels ranged from a 0- to 1-standard-deviation difference in mean ability levels. The results showed that CATSIB with the regression correction displayed good control over Type 1 error, whereas CATSIB without the regression correction displayed impact-induced Type 1 error inflation. With 500 test takers in each group, power rates were exceptionally high (84% to 99%) for values of DIF at the boundary between moderate and large DIF. For smaller samples of 250 test takers in each group, the corresponding power rates ranged from 47% to 95%. In addition, in all cases, CATSIB was very accurate in estimating the true values of DIF, displaying at most only minor estimation bias. (PsycINFO Database Record (c) 2007 APA, all rights reserved) %B Journal of Educational and Behavioral Statistics %I American Educational Research Assn: US %V 29 %P 177-199 %@ 1076-9986 (Print) %G eng %M 2004-19188-002 %0 Report %D 2001 %T CATSIB: A modified SIBTEST procedure to detect differential item functioning in computerized adaptive tests (Research report) %A Nandakumar, R. %A Roussos, L. %I Law School Admission Council %C Newton, PA %G eng %0 Generic %D 1997 %T CATSIB: A modified SIBTEST procedure to detect differential item functioning in computerized adaptive tests (Research report) %A Nandakumar, R. %A Roussos, L. %C Newtown, PA: Law School Admission Council %G eng %0 Conference Paper %B annual meeting of the American Educational Research Association %D 1997 %T Validation of CATSIB To investigate DIF of CAT data %A Nandakumar, R. %A Roussos, L. A. %K computerized adaptive testing %X This paper investigates the performance of CATSIB (a modified version of the SIBTEST computer program) to assess differential item functioning (DIF) in the context of computerized adaptive testing (CAT). One of the distinguishing features of CATSIB is its theoretically built-in regression correction to control for the Type I error rates when the distributions of the reference and focal groups differ on the intended ability. This phenomenon is also called impact. The Type I error rate of CATSIB with the regression correction (WRC) was compared with that of CATSIB without the regression correction (WORC) to see if the regression correction was indeed effective. Also of interest was the power level of CATSIB after the regression correction. The subtest size was set at 25 items, and sample size, the impact level, and the amount of DIF were varied. Results show that the regression correction was very useful in controlling for the Type I error, CATSIB WORC had inflated observed Type I errors, especially when impact levels were high. The CATSIB WRC had observed Type I error rates very close to the nominal level of 0.05. The power rates of CATSIB WRC were impressive. As expected, the power increased as the sample size increased and as the amount of DIF increased. Even for small samples with high impact rates, power rates were 64% or higher for high DIF levels. For large samples, power rates were over 90% for high DIF levels. (Contains 12 tables and 7 references.) (Author/SLD) %B annual meeting of the American Educational Research Association %C Chicago, IL. USA %G eng %M ED409332