%0 Book Section %B Setting performance standards: Concepts, methods, and perspectives %D 2001 %T Practical issues in setting standards on computerized adaptive tests %A Sireci, S. G. %A Clauser, B. E. %K Adaptive Testing %K Computer Assisted Testing %K Performance Tests %K Testing Methods %X (From the chapter) Examples of setting standards on computerized adaptive tests (CATs) are hard to find. Some examples of CATs involving performance standards include the registered nurse exam and the Novell systems engineer exam. Although CATs do not require separate standard setting-methods, there are special issues to be addressed by test specialist who set performance standards on CATs. Setting standards on a CAT will typical require modifications on the procedures used with more traditional, fixed-form, paper-and -pencil examinations. The purpose of this chapter is to illustrate why CATs pose special challenges to the standard setter. (PsycINFO Database Record (c) 2005 APA ) %B Setting performance standards: Concepts, methods, and perspectives %I Lawrence Erlbaum Associates, Inc. %C Mahwah, N.J. USA %P 355-369 %G eng %0 Journal Article %J Dissertation Abstracts International: Section B: The Sciences and Engineering %D 2000 %T An examination of the reliability and validity of performance ratings made using computerized adaptive rating scales %A Buck, D. E. %K Adaptive Testing %K Computer Assisted Testing %K Performance Tests %K Rating Scales %K Reliability %K Test %K Test Validity %X This study compared the psychometric properties of performance ratings made using recently-developed computerized adaptive rating scales (CARS) to the psyc hometric properties of ratings made using more traditional paper-and-pencil rati ng formats, i.e., behaviorally-anchored and graphic rating scales. Specifically, the reliability, validity and accuracy of the performance ratings from each for mat were examined. One hundred twelve participants viewed six 5-minute videotape s of office situations and rated the performance of a target person in each vide otape on three contextual performance dimensions-Personal Support, Organizationa l Support, and Conscientious Initiative-using CARS and either behaviorally-ancho red or graphic rating scales. Performance rating properties were measured using Shrout and Fleiss's intraclass correlation (2, 1), Borman's differential accurac y measure, and Cronbach's accuracy components as indexes of rating reliability, validity, and accuracy, respectively. Results found that performance ratings mad e using the CARS were significantly more reliable and valid than performance rat ings made using either of the other formats. Additionally, CARS yielded more acc urate performance ratings than the paper-and-pencil formats. The nature of the C ARS system (i.e., its adaptive nature and scaling methodology) and its paired co mparison judgment task are offered as possible reasons for the differences found in the psychometric properties of the performance ratings made using the variou s rating formats. (PsycINFO Database Record (c) 2005 APA ) %B Dissertation Abstracts International: Section B: The Sciences and Engineering %V 61 %P 570 %G eng