%0 Journal Article %J Journal of Applied Measurement %D 2008 %T Strategies for controlling item exposure in computerized adaptive testing with the partial credit model %A Davis, L. L. %A Dodd, B. G. %K *Algorithms %K *Computers %K *Educational Measurement/statistics & numerical data %K Humans %K Questionnaires/*standards %K United States %X Exposure control research with polytomous item pools has determined that randomization procedures can be very effective for controlling test security in computerized adaptive testing (CAT). The current study investigated the performance of four procedures for controlling item exposure in a CAT under the partial credit model. In addition to a no exposure control baseline condition, the Kingsbury-Zara, modified-within-.10-logits, Sympson-Hetter, and conditional Sympson-Hetter procedures were implemented to control exposure rates. The Kingsbury-Zara and the modified-within-.10-logits procedures were implemented with 3 and 6 item candidate conditions. The results show that the Kingsbury-Zara and modified-within-.10-logits procedures with 6 item candidates performed as well as the conditional Sympson-Hetter in terms of exposure rates, overlap rates, and pool utilization. These two procedures are strongly recommended for use with partial credit CATs due to their simplicity and strength of their results. %B Journal of Applied Measurement %7 2008/01/09 %V 9 %P 1-17 %@ 1529-7713 (Print)1529-7713 (Linking) %G eng %M 18180546 %0 Generic %D 2005 %T Strategies for controlling item exposure in computerized adaptive testing with the partial credit model %A Davis, L. L. %A Dodd, B. %C Pearson Educational Measurement Research Report 05-01 %G eng %0 Journal Article %J Applied Psychological Measurement %D 2004 %T Strategies for controlling item exposure in computerized adaptive testing with the generalized partial credit model %A Davis, L. L. %K computerized adaptive testing %K generalized partial credit model %K item exposure %X Choosing a strategy for controlling item exposure has become an integral part of test development for computerized adaptive testing (CAT). This study investigated the performance of six procedures for controlling item exposure in a series of simulated CATs under the generalized partial credit model. In addition to a no-exposure control baseline condition, the randomesque, modified-within-.10-logits, Sympson-Hetter, conditional Sympson-Hetter, a-stratified with multiple-stratification, and enhanced a-stratified with multiple-stratification procedures were implemented to control exposure rates. Two variations of the randomesque and modified-within-.10-logits procedures were examined, which varied the size of the item group from which the next item to be administered was randomly selected. The results indicate that although the conditional Sympson-Hetter provides somewhat lower maximum exposure rates, the randomesque and modified-within-.10-logits procedures with the six-item group variation have great utility for controlling overlap rates and increasing pool utilization and should be given further consideration. (PsycINFO Database Record (c) 2007 APA, all rights reserved) %B Applied Psychological Measurement %I Sage Publications: US %V 28 %P 165-185 %@ 0146-6216 (Print) %G eng %M 2004-13800-002 %0 Conference Paper %B Paper presented at the annual meeting of the American Educational Research Association %D 2003 %T A comparison of item exposure control procedures using a CAT system based on the generalized partial credit model %A Burt, W. M %A Kim, S.-J %A Davis, L. L. %A Dodd, B. G. %B Paper presented at the annual meeting of the American Educational Research Association %C Chicago IL %G eng %0 Journal Article %J Journal of Applied Measurement %D 2003 %T An examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model %A Davis, L. L. %A Pastor, D. A. %A Dodd, B. G. %A Chiang, C. %A Fitzpatrick, S. J. %K *Computers %K *Educational Measurement %K *Models, Theoretical %K Automation %K Decision Making %K Humans %K Reproducibility of Results %X The purpose of the present investigation was to systematically examine the effectiveness of the Sympson-Hetter technique and rotated content balancing relative to no exposure control and no content rotation conditions in a computerized adaptive testing system (CAT) based on the partial credit model. A series of simulated fixed and variable length CATs were run using two data sets generated to multiple content areas for three sizes of item pools. The 2 (exposure control) X 2 (content rotation) X 2 (test length) X 3 (item pool size) X 2 (data sets) yielded a total of 48 conditions. Results show that while both procedures can be used with no deleterious effect on measurement precision, the gains in exposure control, pool utilization, and item overlap appear quite modest. Difficulties involved with setting the exposure control parameters in small item pools make questionable the utility of the Sympson-Hetter technique with similar item pools. %B Journal of Applied Measurement %V 4 %P 24-42 %G eng %M 12700429 %0 Journal Article %J Applied Psychological Measurement %D 2003 %T Item exposure constraints for testlets in the verbal reasoning section of the MCAT %A Davis, L. L. %A Dodd, B. G. %K Adaptive Testing %K Computer Assisted Testing %K Entrance Examinations %K Item Response Theory %K Random Sampling %K Reasoning %K Verbal Ability computerized adaptive testing %X The current study examined item exposure control procedures for testlet scored reading passages in the Verbal Reasoning section of the Medical College Admission Test with four computerized adaptive testing (CAT) systems using the partial credit model. The first system used a traditional CAT using maximum information item selection. The second used random item selection to provide a baseline for optimal exposure rates. The third used a variation of Lunz and Stahl's randomization procedure. The fourth used Luecht and Nungester's computerized adaptive sequential testing (CAST) system. A series of simulated fixed-length CATs was run to determine the optimal item length selection procedure. Results indicated that both the randomization procedure and CAST performed well in terms of exposure control and measurement precision, with the CAST system providing the best overall solution when all variables were taken into consideration. (PsycINFO Database Record (c) 2005 APA ) (journal abstract) %B Applied Psychological Measurement %V 27 %P 335-356 %G eng %0 Conference Paper %B Paper presented at the annual meeting of the National Council on Measurement in Education %D 2003 %T Strategies for controlling item exposure in computerized adaptive testing with the generalized partial credit model %A Davis, L. L. %B Paper presented at the annual meeting of the National Council on Measurement in Education %C Chicago IL %G eng %0 Journal Article %J Dissertation Abstracts International: Section B: The Sciences & Engineering %D 2003 %T Strategies for controlling item exposure in computerized adaptive testing with polytomously scored items %A Davis, L. L. %X Choosing a strategy for controlling the exposure of items to examinees has become an integral part of test development for computerized adaptive testing (CAT). Item exposure can be controlled through the use of a variety of algorithms which modify the CAT item selection process. This may be done through a randomization, conditional selection, or stratification approach. The effectiveness of each procedure as well as the degree to which measurement precision is sacrificed has been extensively studied with dichotomously scored item pools. However, only recently have researchers begun to examine these procedures in polytomously scored item pools. The current study investigated the performance of six different exposure control mechanisms under three polytomous IRT models in terms of measurement precision, test security, and ease of implementation. The three models examined in the current study were the partial credit, generalized partial credit, and graded response models. In addition to a no exposure control baseline condition, the randomesque, within .10 logits, Sympson-Hetter, conditional Sympson-Hetter, a-Stratified, and enhanced a-Stratified procedures were implemented to control item exposure rates. The a-Stratified and enhanced a-Stratified procedures were not evaluated with the partial credit model. Two variations of the randomesque and within .10 logits procedures were also examined which varied the size of the item group from which the next item to be administered was randomly selected. The results of this study were remarkably similar for all three models and indicated that the randomesque and within .10 logits procedures, when implemented with the six item group variation, provide the best option for controlling exposure rates when impact to measurement precision and ease of implementation are considered. The three item group variations of the procedures were, however, ineffective in controlling exposure, overlap, and pool utilization rates to desired levels. The Sympson-Hetter and conditional Sympson-Hetter procedures were difficult and time consuming to implement, and while they did control exposure rates to the target level, their performance in terms of item overlap (for the Sympson-Hetter) and pool utilization were disappointing. The a-Stratified and enhanced a-Stratified procedures both turned in surprisingly poor performances across all variables. (PsycINFO Database Record (c) 2004 APA, all rights reserved). %B Dissertation Abstracts International: Section B: The Sciences & Engineering %V 64 %P 458 %G eng %0 Book %D 2002 %T Strategies for controlling item exposure in computerized adaptive testing with polytomously scored items %A Davis, L. L. %C Unpublished doctoral dissertation, University of Texas, Austin %G eng %0 Conference Paper %D 2001 %T An examination of testlet scoring and item exposure constraints in the Verbal Reasoning section of the MCAT %A Davis, L. L. %A Dodd, B. G. %G eng %0 Generic %D 2001 %T An examination of testlet scoring and item exposure constraints in the verbal reasoning section of the MCAT %A Davis, L. L. %A Dodd, B. G. %C MCAT Monograph Series: Association of American Medical Colleges %G eng %0 Conference Paper %B Paper presented at the annual meeting of the American Educational Research Association %D 2000 %T An examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model %A Davis, L. L. %A Pastor, D. A. %A Dodd, B. G. %A Chiang, C. %A Fitzpatrick, S. %B Paper presented at the annual meeting of the American Educational Research Association %C New Orleans, LA %G eng