TY - JOUR T1 - A new stopping rule for computerized adaptive testing JF - Educational and Psychological Measurement Y1 - 2010 A1 - Choi, S. W. A1 - Grady, M. W. A1 - Dodd, B. G. AB - The goal of the current study was to introduce a new stopping rule for computerized adaptive testing. The predicted standard error reduction stopping rule (PSER) uses the predictive posterior variance to determine the reduction in standard error that would result from the administration of additional items. The performance of the PSER was compared to that of the minimum standard error stopping rule and a modified version of the minimum information stopping rule in a series of simulated adaptive tests, drawn from a number of item pools. Results indicate that the PSER makes efficient use of CAT item pools, administering fewer items when predictive gains in information are small and increasing measurement precision when information is abundant. VL - 70 SN - 0013-1644 (Print)0013-1644 (Linking) N1 - U01 AR052177-04/NIAMS NIH HHS/Educ Psychol Meas. 2010 Dec 1;70(6):1-17. U2 - 3028267 ER - TY - JOUR T1 - Strategies for controlling item exposure in computerized adaptive testing with the partial credit model JF - Journal of Applied Measurement Y1 - 2008 A1 - Davis, L. L. A1 - Dodd, B. G. KW - *Algorithms KW - *Computers KW - *Educational Measurement/statistics & numerical data KW - Humans KW - Questionnaires/*standards KW - United States AB - Exposure control research with polytomous item pools has determined that randomization procedures can be very effective for controlling test security in computerized adaptive testing (CAT). The current study investigated the performance of four procedures for controlling item exposure in a CAT under the partial credit model. In addition to a no exposure control baseline condition, the Kingsbury-Zara, modified-within-.10-logits, Sympson-Hetter, and conditional Sympson-Hetter procedures were implemented to control exposure rates. The Kingsbury-Zara and the modified-within-.10-logits procedures were implemented with 3 and 6 item candidate conditions. The results show that the Kingsbury-Zara and modified-within-.10-logits procedures with 6 item candidates performed as well as the conditional Sympson-Hetter in terms of exposure rates, overlap rates, and pool utilization. These two procedures are strongly recommended for use with partial credit CATs due to their simplicity and strength of their results. VL - 9 SN - 1529-7713 (Print)1529-7713 (Linking) N1 - Davis, Laurie LaughlinDodd, Barbara GUnited StatesJournal of applied measurementJ Appl Meas. 2008;9(1):1-17. ER - TY - CONF T1 - A variant of the progressive restricted item exposure control procedure in computerized adaptive testing systems based on the 3PL and the partial credit model T2 - Paper presented at the annual meetings of the American Educational Research Association Y1 - 2006 A1 - McClarty, L. K. A1 - Sperling, R. A1 - Dodd, B. G. JF - Paper presented at the annual meetings of the American Educational Research Association CY - San Francisco ER - TY - JOUR T1 - Computerized adaptive testing with the partial credit model: Estimation procedures, population distributions, and item pool characteristics JF - Applied Psychological Measurement Y1 - 2005 A1 - Gorin, J. A1 - Dodd, B. G. A1 - Fitzpatrick, S. J. A1 - Shieh, Y. Y. VL - 29 ER - TY - CONF T1 - A comparison of exposure control procedures in CAT systems based on different measurement models for testlets using the verbal reasoning section of the MCAT T2 - Paper presented at the Annual meeting of the National Council on Measurement in Education Y1 - 2003 A1 - Boyd, A. M A1 - Dodd, B. G. A1 - Fitzpatrick, S. J. JF - Paper presented at the Annual meeting of the National Council on Measurement in Education CY - Chicago IL N1 - {PDF file, 405 KB} ER - TY - CONF T1 - A comparison of item exposure control procedures using a CAT system based on the generalized partial credit model T2 - Paper presented at the annual meeting of the American Educational Research Association Y1 - 2003 A1 - Burt, W. M A1 - Kim, S.-J A1 - Davis, L. L. A1 - Dodd, B. G. JF - Paper presented at the annual meeting of the American Educational Research Association CY - Chicago IL N1 - {PDF file, 265 KB} ER - TY - JOUR T1 - An examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model JF - Journal of Applied Measurement Y1 - 2003 A1 - Davis, L. L. A1 - Pastor, D. A. A1 - Dodd, B. G. A1 - Chiang, C. A1 - Fitzpatrick, S. J. KW - *Computers KW - *Educational Measurement KW - *Models, Theoretical KW - Automation KW - Decision Making KW - Humans KW - Reproducibility of Results AB - The purpose of the present investigation was to systematically examine the effectiveness of the Sympson-Hetter technique and rotated content balancing relative to no exposure control and no content rotation conditions in a computerized adaptive testing system (CAT) based on the partial credit model. A series of simulated fixed and variable length CATs were run using two data sets generated to multiple content areas for three sizes of item pools. The 2 (exposure control) X 2 (content rotation) X 2 (test length) X 3 (item pool size) X 2 (data sets) yielded a total of 48 conditions. Results show that while both procedures can be used with no deleterious effect on measurement precision, the gains in exposure control, pool utilization, and item overlap appear quite modest. Difficulties involved with setting the exposure control parameters in small item pools make questionable the utility of the Sympson-Hetter technique with similar item pools. VL - 4 N1 - 1529-7713Journal Article ER - TY - JOUR T1 - Item exposure constraints for testlets in the verbal reasoning section of the MCAT JF - Applied Psychological Measurement Y1 - 2003 A1 - Davis, L. L. A1 - Dodd, B. G. KW - Adaptive Testing KW - Computer Assisted Testing KW - Entrance Examinations KW - Item Response Theory KW - Random Sampling KW - Reasoning KW - Verbal Ability computerized adaptive testing AB - The current study examined item exposure control procedures for testlet scored reading passages in the Verbal Reasoning section of the Medical College Admission Test with four computerized adaptive testing (CAT) systems using the partial credit model. The first system used a traditional CAT using maximum information item selection. The second used random item selection to provide a baseline for optimal exposure rates. The third used a variation of Lunz and Stahl's randomization procedure. The fourth used Luecht and Nungester's computerized adaptive sequential testing (CAST) system. A series of simulated fixed-length CATs was run to determine the optimal item length selection procedure. Results indicated that both the randomization procedure and CAST performed well in terms of exposure control and measurement precision, with the CAST system providing the best overall solution when all variables were taken into consideration. (PsycINFO Database Record (c) 2005 APA ) (journal abstract) VL - 27 ER - TY - JOUR T1 - A comparison of item selection techniques and exposure control mechanisms in CATs using the generalized partial credit model JF - Applied Psychological Measurement Y1 - 2002 A1 - Pastor, D. A. A1 - Dodd, B. G. A1 - Chang, Hua-Hua KW - (Statistical) KW - Adaptive Testing KW - Algorithms computerized adaptive testing KW - Computer Assisted Testing KW - Item Analysis KW - Item Response Theory KW - Mathematical Modeling AB - The use of more performance items in large-scale testing has led to an increase in the research investigating the use of polytomously scored items in computer adaptive testing (CAT). Because this research has to be complemented with information pertaining to exposure control, the present research investigated the impact of using five different exposure control algorithms in two sized item pools calibrated using the generalized partial credit model. The results of the simulation study indicated that the a-stratified design, in comparison to a no-exposure control condition, could be used to reduce item exposure and overlap, increase pool utilization, and only minorly degrade measurement precision. Use of the more restrictive exposure control algorithms, such as the Sympson-Hetter and conditional Sympson-Hetter, controlled exposure to a greater extent but at the cost of measurement precision. Because convergence of the exposure control parameters was problematic for some of the more restrictive exposure control algorithms, use of the more simplistic exposure control mechanisms, particularly when the test length to item pool size ratio is large, is recommended. (PsycINFO Database Record (c) 2005 APA ) (journal abstract) VL - 26 ER - TY - ABST T1 - An examination of testlet scoring and item exposure constraints in the verbal reasoning section of the MCAT Y1 - 2001 A1 - Davis, L. L. A1 - Dodd, B. G. CY - MCAT Monograph Series: Association of American Medical Colleges ER - TY - CONF T1 - An examination of testlet scoring and item exposure constraints in the Verbal Reasoning section of the MCAT Y1 - 2001 A1 - Davis, L. L. A1 - Dodd, B. G. N1 - {PDF file, 653 KB} ER - TY - CONF T1 - An investigation of the impact of items that exhibit mild DIF on ability estimation in CAT T2 - Paper presented at the annual meeting of the National Council on Measurement in Education Y1 - 2001 A1 - Jennings, J. A. A1 - Dodd, B. G. A1 - Fitzpatrick, S. JF - Paper presented at the annual meeting of the National Council on Measurement in Education CY - Seattle WA ER - TY - CONF T1 - An examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model T2 - Paper presented at the annual meeting of the American Educational Research Association Y1 - 2000 A1 - Davis, L. L. A1 - Pastor, D. A. A1 - Dodd, B. G. A1 - Chiang, C. A1 - Fitzpatrick, S. JF - Paper presented at the annual meeting of the American Educational Research Association CY - New Orleans, LA ER - TY - CONF T1 - Performance of the Sympson-Hetter exposure control algorithm with a polytomous item bank T2 - Paper presented at the annual meeting of American Educational Research Association Y1 - 1999 A1 - Pastor, D. A. A1 - Chiang, C. A1 - Dodd, B. G. A1 - Yockey, R. and JF - Paper presented at the annual meeting of American Educational Research Association CY - Montreal, Canada ER - TY - CHAP T1 - Alternatives for scoring computerized adaptive tests T2 - Computer-based testing Y1 - 1998 A1 - Dodd, B. G. A1 - Fitzpatrick, S. J. ED - J. J. Fremer ED - W. C. Ward JF - Computer-based testing PB - Lawrence Erlbaum Associates, Inc. CY - Mahwah, N.J., USA ER - TY - CONF T1 - Alternatives for scoring computerized adaptive tests T2 - Paper presented at an Educational Testing Service-sponsored colloquium entitled Computer-based testing: Building the foundations for future assessments Y1 - 1998 A1 - Dodd, B. G. A1 - Fitzpatrick, S. J. JF - Paper presented at an Educational Testing Service-sponsored colloquium entitled Computer-based testing: Building the foundations for future assessments CY - Philadelphia PA ER - TY - JOUR T1 - A comparison of maximum likelihood estimation and expected a posteriori estimation in CAT using the partial credit model JF - Educational and Psychological Measurement Y1 - 1998 A1 - Chen, S. A1 - Hou, L. A1 - Dodd, B. G. VL - 58 ER - TY - JOUR T1 - The effect of population distribution and method of theta estimation on computerized adaptive testing (CAT) using the rating scale model JF - Educational & Psychological Measurement Y1 - 1997 A1 - Chen, S-K. A1 - Hou, L. Y. A1 - Fitzpatrick, S. J. A1 - Dodd, B. G. KW - computerized adaptive testing AB - Investigated the effect of population distribution on maximum likelihood estimation (MLE) and expected a posteriori estimation (EAP) in a simulation study of computerized adaptive testing (CAT) based on D. Andrich's (1978) rating scale model. Comparisons were made among MLE and EAP with a normal prior distribution and EAP with a uniform prior distribution within 2 data sets: one generated using a normal trait distribution and the other using a negatively skewed trait distribution. Descriptive statistics, correlations, scattergrams, and accuracy indices were used to compare the different methods of trait estimation. The EAP estimation with a normal prior or uniform prior yielded results similar to those obtained with MLE, even though the prior did not match the underlying trait distribution. An additional simulation study based on real data suggested that more work is needed to determine the optimal number of quadrature points for EAP in CAT based on the rating scale model. The choice between MLE and EAP for particular measurement situations is discussed. (PsycINFO Database Record (c) 2003 APA, all rights reserved). VL - 57 N1 - Sage Publications, US ER - TY - JOUR T1 - Computerized Adaptive Testing With Polytomous Items JF - Applied Psychological Measurement Y1 - 1995 A1 - Dodd, B. G. A1 - De Ayala, R. J. A1 - Koch. W.R., VL - 19 IS - 1 ER - TY - JOUR T1 - Computerized adaptive testing with polytomous items JF - Applied Psychological Measurement Y1 - 1995 A1 - Dodd, B. G. A1 - De Ayala, R. J., A1 - Koch, W. R. AB - Discusses polytomous item response theory models and the research that has been conducted to investigate a variety of possible operational procedures (item bank, item selection, trait estimation, stopping rule) for polytomous model-based computerized adaptive testing (PCAT). Studies are reviewed that compared PCAT systems based on competing item response theory models that are appropriate for the same measurement objective, as well as applications of PCAT in marketing and educational psychology. Directions for future research using PCAT are suggested. VL - 19 ER - TY - CONF T1 - The effect of population distribution and methods of theta estimation on CAT using the rating scale model T2 - Paper presented at the annual meeting of the American Educational Research Association Y1 - 1995 A1 - Chen, S. A1 - Hou, L. A1 - Fitzpatrick, S. J. A1 - Dodd, B. G. JF - Paper presented at the annual meeting of the American Educational Research Association CY - San Francisco ER - TY - JOUR T1 - An investigation of procedures for computerized adaptive testing using the successive intervals Rasch model JF - Educational and Psychological Measurement Y1 - 1995 A1 - Koch, W. R. A1 - Dodd, B. G. VL - 55 ER - TY - JOUR T1 - Computerized adaptive testing using the partial credit model: Effects of item pool characteristics and different stopping rules JF - Educational and Psychological Measurement Y1 - 1993 A1 - Dodd, B. G. A1 - Koch, W. R. A1 - De Ayala, R. J., AB - Simulated datasets were used to research the effects of the systematic variation of three major variables on the performance of computerized adaptive testing (CAT) procedures for the partial credit model. The three variables studied were the stopping rule for terminating the CATs, item pool size, and the distribution of the difficulty of the items in the pool. Results indicated that the standard error stopping rule performed better across the variety of CAT conditions than the minimum information stopping rule. In addition it was found that item pools that consisted of as few as 30 items were adequate for CAT provided that the item pool was of medium difficulty. The implications of these findings for implementing CAT systems based on the partial credit model are discussed. VL - 53 ER - TY - JOUR T1 - A comparison of the partial credit and graded response models in computerized adaptive testing JF - Applied Measurement in Education Y1 - 1992 A1 - De Ayala, R. J. A1 - Dodd, B. G. A1 - Koch, W. R. VL - 5 ER - TY - JOUR T1 - Computerized adaptive measurement of attitudes JF - Measurement and Evaluation in Counseling and Development Y1 - 1990 A1 - Koch, W. R. A1 - Dodd, B. G. A1 - Fitzpatrick, S. J. VL - 23 ER - TY - JOUR T1 - The effect of item selection procedure and stepsize on computerized adaptive attitude measurement using the rating scale model JF - Applied Psychological Measurement Y1 - 1990 A1 - Dodd, B. G. AB - Real and simulated datasets were used to investigate the effects of the systematic variation of two major variables on the operating characteristics of computerized adaptive testing (CAT) applied to instruments consisting of poly- chotomously scored rating scale items. The two variables studied were the item selection procedure and the stepsize method used until maximum likelihood trait estimates could be calculated. The findings suggested that (1) item pools that consist of as few as 25 items may be adequate for CAT; (2) the variable stepsize method of preliminary trait estimation produced fewer cases of nonconvergence than the use of a fixed stepsize procedure; and (3) the scale value item selection procedure used in conjunction with a minimum standard error stopping rule outperformed the information item selection technique used in conjunction with a minimum information stopping rule in terms of the frequencies of nonconvergent cases, the number of items administered, and the correlations of CAT 0 estimates with full scale estimates and known 0 values. The implications of these findings for implementing CAT with rating scale items are discussed. Index terms: VL - 14 ER - TY - JOUR T1 - The Effect of Item Selection Procedure and Stepsize on Computerized Adaptive Attitude Measurement Using the Rating Scale Model JF - Applied Psychological Measurement Y1 - 1990 A1 - Dodd, B. G. VL - 14 IS - 4 ER - TY - JOUR T1 - A simulation and comparison of flexilevel and Bayesian computerized adaptive testing JF - Journal of Educational Measurement Y1 - 1990 A1 - De Ayala, R. J., A1 - Dodd, B. G. A1 - Koch, W. R. KW - computerized adaptive testing AB - Computerized adaptive testing (CAT) is a testing procedure that adapts an examination to an examinee's ability by administering only items of appropriate difficulty for the examinee. In this study, the authors compared Lord's flexilevel testing procedure (flexilevel CAT) with an item response theory-based CAT using Bayesian estimation of ability (Bayesian CAT). Three flexilevel CATs, which differed in test length (36, 18, and 11 items), and three Bayesian CATs were simulated; the Bayesian CATs differed from one another in the standard error of estimate (SEE) used for terminating the test (0.25, 0.10, and 0.05). Results showed that the flexilevel 36- and 18-item CATs produced ability estimates that may be considered as accurate as those of the Bayesian CAT with SEE = 0.10 and comparable to the Bayesian CAT with SEE = 0.05. The authors discuss the implications for classroom testing and for item response theory-based CAT. VL - 27 ER - TY - JOUR T1 - An investigation of procedures for computerized adaptive testing using partial credit scoring JF - Applied Measurement in Education Y1 - 1989 A1 - Koch, W. R. A1 - Dodd, B. G. VL - 2 ER - TY - JOUR T1 - Operational characteristics of adaptive testing procedures using the graded response model JF - Applied Psychological Measurement Y1 - 1989 A1 - Dodd, B. G. A1 - Koch, W. R. A1 - De Ayala, R. J., VL - 13 ER - TY - JOUR T1 - Operational Characteristics of Adaptive Testing Procedures Using the Graded Response Model JF - Applied Psychological Measurement Y1 - 1989 A1 - Dodd, B. G. A1 - Koch, W. R. A1 - De Ayala, R. J. VL - 13 IS - 2 ER - TY - CONF T1 - Computerized adaptive attitude measurement: A comparison of the graded response and rating scale models T2 - Paper presented at the annual meeting of the American Educational Research Association Y1 - 1988 A1 - Dodd, B. G. A1 - Koch, W. R. A1 - De Ayala, R. J., JF - Paper presented at the annual meeting of the American Educational Research Association CY - New Orleans ER - TY - CONF T1 - Computerized adaptive testing with the rating scale model T2 - Paper presented at the Fourth International Objective Measurement Workshop Y1 - 1987 A1 - Dodd, B. G. JF - Paper presented at the Fourth International Objective Measurement Workshop CY - Chicago ER - TY - CONF T1 - Computerized adaptive attitude measurement T2 - Paper presented at the annual meeting of the American Educational Research Association Y1 - 1985 A1 - Koch, W. R. A1 - Dodd, B. G. JF - Paper presented at the annual meeting of the American Educational Research Association CY - Chicago ER -