01409nas a2200157 4500008003900000245009300039210006900132300001000201490000700211520091300218100002301131700001501154700001701169700001301186856005201199 2013 d00aThe Influence of Item Calibration Error on Variable-Length Computerized Adaptive Testing0 aInfluence of Item Calibration Error on VariableLength Computeriz a24-400 v373 a
Variable-length computerized adaptive testing (VL-CAT) allows both items and test length to be “tailored” to examinees, thereby achieving the measurement goal (e.g., scoring precision or classification) with as few items as possible. Several popular test termination rules depend on the standard error of the ability estimate, which in turn depends on the item parameter values. However, items are chosen on the basis of their parameter estimates, and capitalization on chance may occur. In this article, the authors investigated the effects of capitalization on chance on test length and classification accuracy in several VL-CAT simulations. The results confirm that capitalization on chance occurs in VL-CAT and has complex effects on test length, ability estimation, and classification accuracy. These results have important implications for the design and implementation of VL-CATs.
1 aPatton, Jeffrey, M1 aYing Cheng1 aYuan, Ke-Hai1 aDiao, Qi uhttp://apm.sagepub.com/content/37/1/24.abstract01417nas a2200121 4500008003900000245015400039210006900193300001200262490000700274520094600281100001501227856005301242 2010 d00aImproving Cognitive Diagnostic Computerized Adaptive Testing by Balancing Attribute Coverage: The Modified Maximum Global Discrimination Index Method0 aImproving Cognitive Diagnostic Computerized Adaptive Testing by a902-9130 v703 aThis article proposes a new item selection method, namely, the modified maximum global discrimination index (MMGDI) method, for cognitive diagnostic computerized adaptive testing (CD-CAT). The new method captures two aspects of the appeal of an item: (a) the amount of contribution it can make toward adequate coverage of every attribute and (b) the amount of contribution it can make toward recovering the latent cognitive profile. A simulation study shows that the new method ensures adequate coverage of every attribute, which improves the validity of the test scores, and defensibility of the proposed uses of the test. Furthermore, compared with the original global discrimination index method, the MMGDI method improves the recovery rate of each attribute and of the entire cognitive profile, especially the latter. Therefore, the new method improves both the validity and reliability of the test scores from a CD-CAT program.
1 aYing Cheng uhttp://epm.sagepub.com/content/70/6/902.abstract01651nas a2200157 4500008003900000245010700039210006900146300001000215490000700225520113900232100001501371700001901386700002101405700001501426856005201441 2009 d00aConstraint-Weighted a-Stratification for Computerized Adaptive Testing With Nonstatistical Constraints0 aConstraintWeighted aStratification for Computerized Adaptive Tes a35-490 v693 aa-stratification is a method that utilizes items with small discrimination (a) parameters early in an exam and those with higher a values when more is learned about the ability parameter. It can achieve much better item usage than the maximum information criterion (MIC). To make a-stratification more practical and more widely applicable, a method for weighting the item selection process in a-stratification as a means of satisfying multiple test constraints is proposed. This method is studied in simulation against an analogous method without stratification as well as a-stratification using descending-rather than ascending-a procedures. In addition, a variation of a-stratification that allows for unbalanced usage of a parameters is included in the study to examine the trade-off between efficiency and exposure control. Finally, MIC and randomized item selection are included as baseline measures. Results indicate that the weighting mechanism successfully addresses the constraints, that stratification helps to a great extent balancing exposure rates, and that the ascending-a design improves measurement precision.
1 aYing Cheng1 aChang, Hua-Hua1 aDouglas, Jeffrey1 aFanmin Guo uhttp://epm.sagepub.com/content/69/1/35.abstract01435nas a2200145 4500008003900000245007700039210006900116300001200185490000700197520098600204100001501190700001901205700001201224856005301236 2007 d00aTwo-Phase Item Selection Procedure for Flexible Content Balancing in CAT0 aTwoPhase Item Selection Procedure for Flexible Content Balancing a467-4820 v313 aContent balancing is an important issue in the design and implementation of computerized adaptive testing (CAT). Content-balancing techniques that have been applied in fixed content balancing, where the number of items from each content area is fixed, include constrained CAT (CCAT), the modified multinomial model (MMM), modified constrained CAT (MCCAT), and others. In this article, four methods are proposed to address the flexible content-balancing issue with the a-stratification design, named STR_C. The four methods are MMM+, an extension of MMM; MCCAT+, an extension of MCCAT; the TPM method, a two-phase content-balancing method using MMM in both phases; and the TPF method, a two-phase content-balancing method using MMM in the first phase and MCCAT in the second. Simulation results show that all of the methods work well in content balancing, and TPF performs the best in item exposure control and item pool utilization while maintaining measurement precision.
1 aYing Cheng1 aChang, Hua-Hua1 aQing Yi uhttp://apm.sagepub.com/content/31/6/467.abstract