01889nas a2200157 4500008003900000245006900039210006800108300001200176490000700188520141000195100001901605700001601624700002001640700001801660856005301678 2014 d00aGeneral Test Overlap Control: Improved Algorithm for CAT and CCT0 aGeneral Test Overlap Control Improved Algorithm for CAT and CCT a229-2440 v383 a

This article proposed a new online test overlap control algorithm that is an improvement of Chen’s algorithm in controlling general test overlap rate for item pooling among a group of examinees. Chen’s algorithm is not very efficient in that not only item pooling between current examinee and prior examinees is controlled for but also item pooling between previous examinees, which would have been controlled for when they were current examinees. The proposed improvement increases efficiency by only considering item pooling between current and previous examinees, and its improved performance over Chen is demonstrated in a simulated computerized adaptive testing (CAT) environment. Moreover, the proposed algorithm is adapted for computerized classification testing (CCT) using the sequential probability ratio test procedure and is evaluated against some existing exposure control procedures. The proposed algorithm appears to work best in controlling general test overlap rate among the exposure control procedures examined without sacrificing much classification precision, though longer tests might be required for more stringent control of item pooling among larger groups. Given the capability of the proposed algorithm in controlling item pooling among a group of examinees of any size and its ease of implementation, it appears to be a good test overlap control method.

1 aChen, Shu-Ying1 aLei, Pui-Wa1 aChen, Jyun-Hong1 aLiu, Tzu-Chen uhttp://apm.sagepub.com/content/38/3/229.abstract01526nas a2200157 4500008003900000022001400039245011200053210006900165300001400234490000700248520101100255100001601266700001901282700001201301856005501313 2006 d a1745-398400aComparing Methods of Assessing Differential Item Functioning in a Computerized Adaptive Testing Environment0 aComparing Methods of Assessing Differential Item Functioning in  a245–2640 v433 a

Mantel-Haenszel and SIBTEST, which have known difficulty in detecting non-unidirectional differential item functioning (DIF), have been adapted with some success for computerized adaptive testing (CAT). This study adapts logistic regression (LR) and the item-response-theory-likelihood-ratio test (IRT-LRT), capable of detecting both unidirectional and non-unidirectional DIF, to the CAT environment in which pretest items are assumed to be seeded in CATs but not used for trait estimation. The proposed adaptation methods were evaluated with simulated data under different sample size ratios and impact conditions in terms of Type I error, power, and specificity in identifying the form of DIF. The adapted LR and IRT-LRT procedures are more powerful than the CAT version of SIBTEST for non-unidirectional DIF detection. The good Type I error control provided by IRT-LRT under extremely unequal sample sizes and large impact is encouraging. Implications of these and other findings are discussed.

1 aLei, Pui-Wa1 aChen, Shu-Ying1 aYu, Lan uhttp://dx.doi.org/10.1111/j.1745-3984.2006.00015.x01320nas a2200133 4500008003900000245008000039210006900119300001200188490000700200520089100207100001901098700001601117856005301133 2005 d00aControlling Item Exposure and Test Overlap in Computerized Adaptive Testing0 aControlling Item Exposure and Test Overlap in Computerized Adapt a204-2170 v293 a

This article proposes an item exposure control method, which is the extension of the Sympson and Hetter procedure and can provide item exposure control at both the item and test levels. Item exposure rate and test overlap rate are two indices commonly used to track item exposure in computerized adaptive tests. By considering both indices, item exposure can be monitored at both the item and test levels. To control the item exposure rate and test overlap rate simultaneously, the modified procedure attempted to control not only the maximum value but also the variance of item exposure rates. Results indicated that the item exposure rate and test overlap rate could be controlled simultaneously by implementing the modified procedure. Item exposure control was improved and precision of trait estimation decreased when a prespecified maximum test overlap rate was stringent.

1 aChen, Shu-Ying1 aLei, Pui-Wa uhttp://apm.sagepub.com/content/29/3/204.abstract