%0 Journal Article %J British Journal of Mathematical and Statistical Psychology %D 2009 %T The maximum priority index method for severely constrained item selection in computerized adaptive testing %A Cheng, Y %A Chang, Hua-Hua %K Aptitude Tests/*statistics & numerical data %K Diagnosis, Computer-Assisted/*statistics & numerical data %K Educational Measurement/*statistics & numerical data %K Humans %K Mathematical Computing %K Models, Statistical %K Personality Tests/*statistics & numerical data %K Psychometrics/*statistics & numerical data %K Reproducibility of Results %K Software %X This paper introduces a new heuristic approach, the maximum priority index (MPI) method, for severely constrained item selection in computerized adaptive testing. Our simulation study shows that it is able to accommodate various non-statistical constraints simultaneously, such as content balancing, exposure control, answer key balancing, and so on. Compared with the weighted deviation modelling method, it leads to fewer constraint violations and better exposure control while maintaining the same level of measurement precision. %B British Journal of Mathematical and Statistical Psychology %7 2008/06/07 %V 62 %P 369-83 %8 May %@ 0007-1102 (Print)0007-1102 (Linking) %G eng %M 18534047 %0 Journal Article %J Journal of Applied Measurement %D 2008 %T Binary items and beyond: a simulation of computer adaptive testing using the Rasch partial credit model %A Lange, R. %K *Data Interpretation, Statistical %K *User-Computer Interface %K Educational Measurement/*statistics & numerical data %K Humans %K Illinois %K Models, Statistical %X Past research on Computer Adaptive Testing (CAT) has focused almost exclusively on the use of binary items and minimizing the number of items to be administrated. To address this situation, extensive computer simulations were performed using partial credit items with two, three, four, and five response categories. Other variables manipulated include the number of available items, the number of respondents used to calibrate the items, and various manipulations of respondents' true locations. Three item selection strategies were used, and the theoretically optimal Maximum Information method was compared to random item selection and Bayesian Maximum Falsification approaches. The Rasch partial credit model proved to be quite robust to various imperfections, and systematic distortions did occur mainly in the absence of sufficient numbers of items located near the trait or performance levels of interest. The findings further indicate that having small numbers of items is more problematic in practice than having small numbers of respondents to calibrate these items. Most importantly, increasing the number of response categories consistently improved CAT's efficiency as well as the general quality of the results. In fact, increasing the number of response categories proved to have a greater positive impact than did the choice of item selection method, as the Maximum Information approach performed only slightly better than the Maximum Falsification approach. Accordingly, issues related to the efficiency of item selection methods are far less important than is commonly suggested in the literature. However, being based on computer simulations only, the preceding presumes that actual respondents behave according to the Rasch model. CAT research could thus benefit from empirical studies aimed at determining whether, and if so, how, selection strategies impact performance. %B Journal of Applied Measurement %7 2008/01/09 %V 9 %P 81-104 %@ 1529-7713 (Print)1529-7713 (Linking) %G eng %M 18180552 %0 Journal Article %J Spanish Journal of Psychology %D 2008 %T Rotating item banks versus restriction of maximum exposure rates in computerized adaptive testing %A Barrada, J %A Olea, J. %A Abad, F. J. %K *Character %K *Databases %K *Software Design %K Aptitude Tests/*statistics & numerical data %K Bias (Epidemiology) %K Computing Methodologies %K Diagnosis, Computer-Assisted/*statistics & numerical data %K Educational Measurement/*statistics & numerical data %K Humans %K Mathematical Computing %K Psychometrics/statistics & numerical data %X

If examinees were to know, beforehand, part of the content of a computerized adaptive test, their estimated trait levels would then have a marked positive bias. One of the strategies to avoid this consists of dividing a large item bank into several sub-banks and rotating the sub-bank employed (Ariel, Veldkamp & van der Linden, 2004). This strategy permits substantial improvements in exposure control at little cost to measurement accuracy, However, we do not know whether this option provides better results than using the master bank with greater restriction in the maximum exposure rates (Sympson & Hetter, 1985). In order to investigate this issue, we worked with several simulated banks of 2100 items, comparing them, for RMSE and overlap rate, with the same banks divided in two, three... up to seven sub-banks. By means of extensive manipulation of the maximum exposure rate in each bank, we found that the option of rotating banks slightly outperformed the option of restricting maximum exposure rate of the master bank by means of the Sympson-Hetter method.

%B Spanish Journal of Psychology %7 2008/11/08 %V 11 %P 618-625 %@ 1138-7416 %G eng %M 18988447 %0 Journal Article %J Journal of Applied Measurement %D 2000 %T CAT administration of language placement examinations %A Stahl, J. %A Bergstrom, B. %A Gershon, R. C. %K *Language %K *Software %K Aptitude Tests/*statistics & numerical data %K Educational Measurement/*statistics & numerical data %K Humans %K Psychometrics %K Reproducibility of Results %K Research Support, Non-U.S. Gov't %X This article describes the development of a computerized adaptive test for Cegep de Jonquiere, a community college located in Quebec, Canada. Computerized language proficiency testing allows the simultaneous presentation of sound stimuli as the question is being presented to the test-taker. With a properly calibrated bank of items, the language proficiency test can be offered in an adaptive framework. By adapting the test to the test-taker's level of ability, an assessment can be made with significantly fewer items. We also describe our initial attempt to detect instances in which "cheating low" is occurring. In the "cheating low" situation, test-takers deliberately answer questions incorrectly, questions that they are fully capable of answering correctly had they been taking the test honestly. %B Journal of Applied Measurement %V 1 %P 292-302 %G eng %M 12029172