%0 Journal Article %J Applied Psychological Measurement %D 2016 %T Maximum Likelihood Score Estimation Method With Fences for Short-Length Tests and Computerized Adaptive Tests %A Han, Kyung T. %X A critical shortcoming of the maximum likelihood estimation (MLE) method for test score estimation is that it does not work with certain response patterns, including ones consisting only of all 0s or all 1s. This can be problematic in the early stages of computerized adaptive testing (CAT) administration and for tests short in length. To overcome this challenge, test practitioners often set lower and upper bounds of theta estimation and truncate the score estimation to be one of those bounds when the log likelihood function fails to yield a peak due to responses consisting only of 0s or 1s. Even so, this MLE with truncation (MLET) method still cannot handle response patterns in which all harder items are correct and all easy items are incorrect. Bayesian-based estimation methods such as the modal a posteriori (MAP) method or the expected a posteriori (EAP) method can be viable alternatives to MLE. The MAP or EAP methods, however, are known to result in estimates biased toward the center of a prior distribution, resulting in a shrunken score scale. This study introduces an alternative approach to MLE, called MLE with fences (MLEF). In MLEF, several imaginary “fence” items with fixed responses are introduced to form a workable log likelihood function even with abnormal response patterns. The findings of this study suggest that, unlike MLET, the MLEF can handle any response patterns and, unlike both MAP and EAP, results in score estimates that do not cause shrinkage of the theta scale. %B Applied Psychological Measurement %V 40 %P 289-301 %U http://apm.sagepub.com/content/40/4/289.abstract %R 10.1177/0146621616631317 %0 Journal Article %J Applied Psychological Measurement %D 2013 %T Item Pocket Method to Allow Response Review and Change in Computerized Adaptive Testing %A Han, Kyung T. %X

Most computerized adaptive testing (CAT) programs do not allow test takers to review and change their responses because it could seriously deteriorate the efficiency of measurement and make tests vulnerable to manipulative test-taking strategies. Several modified testing methods have been developed that provide restricted review options while limiting the trade-off in CAT efficiency. The extent to which these methods provided test takers with options to review test items, however, still was quite limited. This study proposes the item pocket (IP) method, a new testing approach that allows test takers greater flexibility in changing their responses by eliminating restrictions that prevent them from moving across test sections to review their answers. A series of simulations were conducted to evaluate the robustness of the IP method against various manipulative test-taking strategies. Findings and implications of the study suggest that the IP method may be an effective solution for many CAT programs when the IP size and test time limit are properly set.

%B Applied Psychological Measurement %V 37 %P 259-275 %U http://apm.sagepub.com/content/37/4/259.abstract %R 10.1177/0146621612473638 %0 Journal Article %J Journal of Educational Measurement %D 2012 %T An Efficiency Balanced Information Criterion for Item Selection in Computerized Adaptive Testing %A Han, Kyung T. %X

Successful administration of computerized adaptive testing (CAT) programs in educational settings requires that test security and item exposure control issues be taken seriously. Developing an item selection algorithm that strikes the right balance between test precision and level of item pool utilization is the key to successful implementation and long-term quality control of CAT. This study proposed a new item selection method using the “efficiency balanced information” criterion to address issues with the maximum Fisher information method and stratification methods. According to the simulation results, the new efficiency balanced information method had desirable advantages over the other studied item selection methods in terms of improving the optimality of CAT assembly and utilizing items with low a-values while eliminating the need for item pool stratification.

%B Journal of Educational Measurement %V 49 %P 225–246 %U http://dx.doi.org/10.1111/j.1745-3984.2012.00173.x %R 10.1111/j.1745-3984.2012.00173.x