TY - JOUR T1 - An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales JF - Journal of Applied Psychology Y1 - 2001 A1 - Borman, W. C. A1 - Buck, D. E. A1 - Hanson, M. A. A1 - Motowidlo, S. J. A1 - Stark, S. A1 - F Drasgow KW - *Computer Simulation KW - *Employee Performance Appraisal KW - *Personnel Selection KW - Adult KW - Automatic Data Processing KW - Female KW - Human KW - Male KW - Reproducibility of Results KW - Sensitivity and Specificity KW - Support, U.S. Gov't, Non-P.H.S. KW - Task Performance and Analysis KW - Video Recording AB - This laboratory research compared the reliability, validity, and accuracy of a computerized adaptive rating scale (CARS) format and 2 relatively common and representative rating formats. The CARS is a paired-comparison rating task that uses adaptive testing principles to present pairs of scaled behavioral statements to the rater to iteratively estimate a ratee's effectiveness on 3 dimensions of contextual performance. Videotaped vignettes of 6 office workers were prepared, depicting prescripted levels of contextual performance, and 112 subjects rated these vignettes using the CARS format and one or the other competing format. Results showed 23%-37% lower standard errors of measurement for the CARS format. In addition, validity was significantly higher for the CARS format (d = .18), and Cronbach's accuracy coefficients showed significantly higher accuracy, with a median effect size of .08. The discussion focuses on possible reasons for the results. VL - 86 N1 - 214803450021-9010Journal ArticleValidation Studies ER -