%0 Book Section %B Handbook of multimethod measurement in psychology %D 2006 %T Computer-based testing %A F Drasgow %A Chuah, S. C. %K Adaptive Testing computerized adaptive testing %K Computer Assisted Testing %K Experimentation %K Psychometrics %K Theories %X (From the chapter) There has been a proliferation of research designed to explore and exploit opportunities provided by computer-based assessment. This chapter provides an overview of the diverse efforts by researchers in this area. It begins by describing how paper-and-pencil tests can be adapted for administration by computers. Computerization provides the important advantage that items can be selected so they are of appropriate difficulty for each examinee. Some of the psychometric theory needed for computerized adaptive testing is reviewed. Then research on innovative computerized assessments is summarized. These assessments go beyond multiple-choice items by using formats made possible by computerization. Then some hardware and software issues are described, and finally, directions for future work are outlined. (PsycINFO Database Record (c) 2006 APA ) %B Handbook of multimethod measurement in psychology %I American Psychological Association %C Washington D.C. USA %V xiv %P 87-100 %G eng %0 Journal Article %J Applied Measurement in Education %D 2006 %T How Big Is Big Enough? Sample Size Requirements for CAST Item Parameter Estimation %A Chuah, Siang Chee %A F Drasgow %A Luecht, Richard %B Applied Measurement in Education %V 19 %P 241-255 %U http://www.tandfonline.com/doi/abs/10.1207/s15324818ame1903_5 %R 10.1207/s15324818ame1903_5 %0 Conference Paper %B Paper presented at the annual meeting of the National Council on Measurement in Education %D 2004 %T Item parameter recovery with adaptive tests %A Do, B.-R. %A Chuah, S. C. %A F Drasgow %B Paper presented at the annual meeting of the National Council on Measurement in Education %C San Diego CA %G eng %0 Journal Article %J Applied Psychological Measurement %D 2002 %T An EM approach to parameter estimation for the Zinnes and Griggs paired comparison IRT model %A Stark, S. %A F Drasgow %K Adaptive Testing %K Computer Assisted Testing %K Item Response Theory %K Maximum Likelihood %K Personnel Evaluation %K Statistical Correlation %K Statistical Estimation %X Borman et al. recently proposed a computer adaptive performance appraisal system called CARS II that utilizes paired comparison judgments of behavioral stimuli. To implement this approach,the paired comparison ideal point model developed by Zinnes and Griggs was selected. In this article,the authors describe item response and information functions for the Zinnes and Griggs model and present procedures for estimating stimulus and person parameters. Monte carlo simulations were conducted to assess the accuracy of the parameter estimation procedures. The results indicated that at least 400 ratees (i.e.,ratings) are required to obtain reasonably accurate estimates of the stimulus parameters and their standard errors. In addition,latent trait estimation improves as test length increases. The implications of these results for test construction are also discussed. %B Applied Psychological Measurement %V 26 %P 208-227 %G eng %0 Book Section %B Computer-based tests: Building the foundation for future assessment %D 2002 %T The work ahead: A psychometric infrastructure for computerized adaptive tests %A F Drasgow %E M. P. Potenza %E J. J. Freemer %E W. C. Ward %K Adaptive Testing %K Computer Assisted Testing %K Educational %K Measurement %K Psychometrics %X (From the chapter) Considers the past and future of computerized adaptive tests and computer-based tests and looks at issues and challenges confronting a testing program as it implements and operates a computer-based test. Recommendations for testing programs from The National Council of Measurement in Education Ad Hoc Committee on Computerized Adaptive Test Disclosure are appended. (PsycINFO Database Record (c) 2005 APA ) %B Computer-based tests: Building the foundation for future assessment %I Lawrence Erlbaum Associates, Inc. %C Mahwah, N.J. USA %G eng %0 Journal Article %J Journal of Applied Psychology %D 2001 %T An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales %A Borman, W. C. %A Buck, D. E. %A Hanson, M. A. %A Motowidlo, S. J. %A Stark, S. %A F Drasgow %K *Computer Simulation %K *Employee Performance Appraisal %K *Personnel Selection %K Adult %K Automatic Data Processing %K Female %K Human %K Male %K Reproducibility of Results %K Sensitivity and Specificity %K Support, U.S. Gov't, Non-P.H.S. %K Task Performance and Analysis %K Video Recording %X This laboratory research compared the reliability, validity, and accuracy of a computerized adaptive rating scale (CARS) format and 2 relatively common and representative rating formats. The CARS is a paired-comparison rating task that uses adaptive testing principles to present pairs of scaled behavioral statements to the rater to iteratively estimate a ratee's effectiveness on 3 dimensions of contextual performance. Videotaped vignettes of 6 office workers were prepared, depicting prescripted levels of contextual performance, and 112 subjects rated these vignettes using the CARS format and one or the other competing format. Results showed 23%-37% lower standard errors of measurement for the CARS format. In addition, validity was significantly higher for the CARS format (d = .18), and Cronbach's accuracy coefficients showed significantly higher accuracy, with a median effect size of .08. The discussion focuses on possible reasons for the results. %B Journal of Applied Psychology %V 86 %P 965-973 %G eng %M 11596812 %0 Book %D 1999 %T Innovations in computerized assessment %A F Drasgow %A Olson-Buchanan, J. B. %K computerized adaptive testing %X Chapters in this book present the challenges and dilemmas faced by researchers as they created new computerized assessments, focusing on issues addressed in developing, scoring, and administering the assessments. Chapters are: (1) "Beyond Bells and Whistles; An Introduction to Computerized Assessment" (Julie B. Olson-Buchanan and Fritz Drasgow); (2) "The Development of a Computerized Selection System for Computer Programmers in a Financial Services Company" (Michael J. Zickar, Randall C. Overton, L. Rogers Taylor, and Harvey J. Harms); (3) "Development of the Computerized Adaptive Testing Version of the Armed Services Vocational Aptitude Battery" (Daniel O. Segall and Kathleen E. Moreno); (4) "CAT for Certification and Licensure" (Betty A. Bergstrom and Mary E. Lunz); (5) "Developing Computerized Adaptive Tests for School Children" (G. Gage Kingsbury and Ronald L. Houser); (6) "Development and Introduction of a Computer Adaptive Graduate Record Examinations General Test" (Craig N. Mills); (7) "Computer Assessment Using Visual Stimuli: A Test of Dermatological Skin Disorders" (Terry A. Ackerman, John Evans, Kwang-Seon Park, Claudia Tamassia, and Ronna Turner); (8) "Creating Computerized Adaptive Tests of Music Aptitude: Problems, Solutions, and Future Directions" (Walter P. Vispoel); (9) "Development of an Interactive Video Assessment: Trials and Tribulations" (Fritz Drasgow, Julie B. Olson-Buchanan, and Philip J. Moberg); (10) "Computerized Assessment of Skill for a Highly Technical Job" (Mary Ann Hanson, Walter C. Borman, Henry J. Mogilka, Carol Manning, and Jerry W. Hedge); (11) "Easing the Implementation of Behavioral Testing through Computerization" (Wayne A. Burroughs, Janet Murray, S. Scott Wesley, Debra R. Medina, Stacy L. Penn, Steven R. Gordon, and Michael Catello); and (12) "Blood, Sweat, and Tears: Some Final Comments on Computerized Assessment." (Fritz Drasgow and Julie B. Olson-Buchanan). Each chapter contains references. (Contains 17 tables and 21 figures.) (SLD) %I Lawrence Erlbaum Associates, Inc. %C Mahwah, N.J. %G eng %M ED435694 %0 Conference Paper %B Paper presented at the annual meeting of the Society for Industrial and Organization Psychology %D 1998 %T Application of an IRT ideal point model to computer adaptive assessment of job performance %A Stark, S. %A F Drasgow %B Paper presented at the annual meeting of the Society for Industrial and Organization Psychology %C Dallas TX %G eng %0 Conference Paper %B Paper presented at the 3th annual conference of the Society for Industrial and Organizational Psychology %D 1998 %T Computerized adaptive rating scales that measure contextual performance %A Borman, W. C. %A Hanson, M. A. %A Montowidlo, S. J %A F Drasgow %A Foster, L %A Kubisiak, U. C. %B Paper presented at the 3th annual conference of the Society for Industrial and Organizational Psychology %C Dallas TX %G eng