TY - JOUR T1 - An Investigation of Exposure Control Methods With Variable-Length CAT Using the Partial Credit Model JF - Applied Psychological Measurement Y1 - 2019 A1 - Audrey J. Leroux A1 - J. Kay Waid-Ebbs A1 - Pey-Shan Wen A1 - Drew A. Helmer A1 - David P. Graham A1 - Maureen K. O’Connor A1 - Kathleen Ray AB - The purpose of this simulation study was to investigate the effect of several different item exposure control procedures in computerized adaptive testing (CAT) with variable-length stopping rules using the partial credit model. Previous simulation studies on CAT exposure control methods with polytomous items rarely considered variable-length tests. The four exposure control techniques examined were the randomesque with a group of three items, randomesque with a group of six items, progressive-restricted standard error (PR-SE), and no exposure control. The two variable-length stopping rules included were the SE and predicted standard error reduction (PSER), along with three item pools of varied sizes (43, 86, and 172 items). Descriptive statistics on number of nonconvergent cases, measurement precision, testing burden, item overlap, item exposure, and pool utilization were calculated. Results revealed that the PSER stopping rule administered fewer items on average while maintaining measurement precision similar to the SE stopping rule across the different item pool sizes and exposure controls. The PR-SE exposure control procedure surpassed the randomesque methods by further reducing test overlap, maintaining maximum exposure rates at the target rate or lower, and utilizing all items from the pool with a minimal increase in number of items administered and nonconvergent cases. VL - 43 UR - https://doi.org/10.1177/0146621618824856 ER - TY - JOUR T1 - The validation of a computer-adaptive test (CAT) for assessing health-related quality of life in children and adolescents in a clinical sample: study design, methods and first results of the Kids-CAT study JF - Quality of Life Research Y1 - 2017 A1 - Barthel, D. A1 - Otto, C. A1 - Nolte, S. A1 - Meyrose, A.-K. A1 - Fischer, F. A1 - Devine, J. A1 - Walter, O. A1 - Mierke, A. A1 - Fischer, K. I. A1 - Thyen, U. A1 - Klein, M. A1 - Ankermann, T. A1 - Rose, M. A1 - Ravens-Sieberer, U. AB - Recently, we developed a computer-adaptive test (CAT) for assessing health-related quality of life (HRQoL) in children and adolescents: the Kids-CAT. It measures five generic HRQoL dimensions. The aims of this article were (1) to present the study design and (2) to investigate its psychometric properties in a clinical setting. VL - 26 UR - https://doi.org/10.1007/s11136-016-1437-9 ER - TY - JOUR T1 - Comparing Simple Scoring With IRT Scoring of Personality Measures: The Navy Computer Adaptive Personality Scales JF - Applied Psychological Measurement Y1 - 2015 A1 - Oswald, Frederick L. A1 - Shaw, Amy A1 - Farmer, William L. AB -

This article analyzes data from U.S. Navy sailors (N = 8,956), with the central measure being the Navy Computer Adaptive Personality Scales (NCAPS). Analyses and results from this article extend and qualify those from previous research efforts by examining the properties of the NCAPS and its adaptive structure in more detail. Specifically, this article examines item exposure rates, the efficiency of item use based on item response theory (IRT)–based Expected A Posteriori (EAP) scoring, and a comparison of IRT-EAP scoring with much more parsimonious scoring methods that appear to work just as well (stem-level scoring and dichotomous scoring). The cutting-edge nature of adaptive personality testing will necessitate a series of future efforts like this: to examine the benefits of adaptive scoring schemes and novel measurement methods continually, while pushing testing technology further ahead.

VL - 39 UR - http://apm.sagepub.com/content/39/2/144.abstract ER - TY - ABST T1 - Cross-cultural development of an item list for computer-adaptive testing of fatigue in oncological patients Y1 - 2011 A1 - Giesinger, J. M. A1 - Petersen, M. A. A1 - Groenvold, M. A1 - Aaronson, N. K. A1 - Arraras, J. I. A1 - Conroy, T. A1 - Gamper, E. M. A1 - Kemmler, G. A1 - King, M. T. A1 - Oberguggenberger, A. S. A1 - Velikova, G. A1 - Young, T. A1 - Holzner, B. A1 - Eortc-Qlg, E. O. AB - ABSTRACT: INTRODUCTION: Within an ongoing project of the EORTC Quality of Life Group, we are developing computerized adaptive test (CAT) measures for the QLQ-C30 scales. These new CAT measures are conceptualised to reflect the same constructs as the QLQ-C30 scales. Accordingly, the Fatigue-CAT is intended to capture physical and general fatigue. METHODS: The EORTC approach to CAT development comprises four phases (literature search, operationalisation, pre-testing, and field testing). Phases I-III are described in detail in this paper. A literature search for fatigue items was performed in major medical databases. After refinement through several expert panels, the remaining items were used as the basis for adapting items and/or formulating new items fitting the EORTC item style. To obtain feedback from patients with cancer, these English items were translated into Danish, French, German, and Spanish and tested in the respective countries. RESULTS: Based on the literature search a list containing 588 items was generated. After a comprehensive item selection procedure focusing on content, redundancy, item clarity and item difficulty a list of 44 fatigue items was generated. Patient interviews (n=52) resulted in 12 revisions of wording and translations. DISCUSSION: The item list developed in phases I-III will be further investigated within a field-testing phase (IV) to examine psychometric characteristics and to fit an item response theory model. The Fatigue CAT based on this item bank will provide scores that are backward-compatible to the original QLQ-C30 fatigue scale. JF - Health and Quality of Life Outcomes VL - 9 SN - 1477-7525 (Electronic)1477-7525 (Linking) N1 - Health Qual Life Outcomes. 2011 Mar 29;9(1):19. ER - TY - JOUR T1 - Deterioro de parámetros de los ítems en tests adaptativos informatizados: estudio con eCAT [Item parameter drift in computerized adaptive testing: Study with eCAT] JF - Psicothema Y1 - 2010 A1 - Abad, F. J. A1 - Olea, J. A1 - Aguado, D. A1 - Ponsoda, V. A1 - Barrada, J KW - *Software KW - Educational Measurement/*methods/*statistics & numerical data KW - Humans KW - Language AB -

En el presente trabajo se muestra el análisis realizado sobre un Test Adaptativo Informatizado (TAI) diseñado para la evaluación del nivel de inglés, denominado eCAT, con el objetivo de estudiar el deterioro de parámetros (parameter drift) producido desde la calibración inicial del banco de ítems. Se ha comparado la calibración original desarrollada para la puesta en servicio del TAI (N= 3224) y la calibración actual obtenida con las aplicaciones reales del TAI (N= 7254). Se ha analizado el Funcionamiento Diferencial de los Ítems (FDI) en función de los parámetros utilizados y se ha simulado el impacto que sobre el nivel de rasgo estimado tiene la variación en los parámetros. Los resultados muestran que se produce especialmente un deterioro de los parámetros a y c, que hay unimportante número de ítems del banco para los que existe FDI y que la variación de los parámetros produce un impacto moderado en la estimación de θ de los evaluados con nivel de inglés alto. Se concluye que los parámetros de los ítems se han deteriorado y deben ser actualizados.Item parameter drift in computerized adaptive testing: Study with eCAT. This study describes the parameter drift analysis conducted on eCAT (a Computerized Adaptive Test to assess the written English level of Spanish speakers). The original calibration of the item bank (N = 3224) was compared to a new calibration obtained from the data provided by most eCAT operative administrations (N =7254). A Differential Item Functioning (DIF) study was conducted between the original and the new calibrations. The impact that the new parameters have on the trait level estimates was obtained by simulation. Results show that parameter drift is found especially for a and c parameters, an important number of bank items show DIF, and the parameter change has a moderate impact on high-level-English θ estimates. It is then recommended to replace the original estimates by the new set. by the new set.

VL - 22 SN - 0214-9915 (Print)0214-9915 (Linking) N1 - Abad, Francisco JOlea, JulioAguado, DavidPonsoda, VicenteBarrada, Juan REnglish AbstractSpainPsicothemaPsicothema. 2010 May;22(2):340-7. ER - TY - JOUR T1 - A Method for the Comparison of Item Selection Rules in Computerized Adaptive Testing JF - Applied Psychological Measurement Y1 - 2010 A1 - Barrada, Juan Ramón A1 - Olea, Julio A1 - Ponsoda, Vicente A1 - Abad, Francisco José AB -

In a typical study comparing the relative efficiency of two item selection rules in computerized adaptive testing, the common result is that they simultaneously differ in accuracy and security, making it difficult to reach a conclusion on which is the more appropriate rule. This study proposes a strategy to conduct a global comparison of two or more selection rules. A plot showing the performance of each selection rule for several maximum exposure rates is obtained and the whole plot is compared with other rule plots. The strategy was applied in a simulation study with fixed-length CATs for the comparison of six item selection rules: the point Fisher information, Fisher information weighted by likelihood, Kullback-Leibler weighted by likelihood, maximum information stratification with blocking, progressive and proportional methods. Our results show that there is no optimal rule for any overlap value or root mean square error (RMSE). The fact that a rule, for a given level of overlap, has lower RMSE than another does not imply that this pattern holds for another overlap rate. A fair comparison of the rules requires extensive manipulation of the maximum exposure rates. The best methods were the Kullback-Leibler weighted by likelihood, the proportional method, and the maximum information stratification method with blocking.

VL - 34 UR - http://apm.sagepub.com/content/34/6/438.abstract ER - TY - JOUR T1 - Tests informatizados y otros nuevos tipos de tests [Computerized and other new types of tests] JF - Papeles del Psicólogo Y1 - 2010 A1 - Olea, J. A1 - Abad, F. J. A1 - Barrada, J AB - Recientemente se ha producido un considerable desarrollo de los tests adaptativos informatizados, en los que el test se adapta progresivamente al rendimiento del evaluando, y de otros tipos de tests: a) los test basados en modelos (se dispone de un modelo o teoría de cómo se responde a cada ítem, lo que permite predecir su dificultad), b) los tests ipsativos (el evaluado ha de elegir entre opciones que tienen parecida deseabilidad social, por lo que pueden resultar eficaces para controlar algunos sesgos de respuestas), c) los tests conductuales (miden rasgos que ordinariamente se han venido midiendo con autoinformes, mediante tareas que requieren respuestas no verbales) y d) los tests situacionales (en los que se presenta al evaluado una situación de conflicto laboral, por ejemplo, con varias posibles soluciones, y ha de elegir la que le parece la mejor descripción de lo que el haría en esa situación). El artículo comenta las características, ventajas e inconvenientes de todos ellos y muestra algunos ejemplos de tests concretos. Palabras clave: Test adaptativo informatizado, Test situacional, Test comportamental, Test ipsativo y generación automática de ítems.The paper provides a short description of some test types that are earning considerable interest in both research and applied areas. The main feature of a computerized adaptive test is that in despite of the examinees receiving different sets of items, their test scores are in the same metric and can be directly compared. Four other test types are considered: a) model-based tests (a model or theory is available to explain the item response process and this makes the prediction of item difficulties possible), b) ipsative tests (the examinee has to select one among two or more options with similar social desirability; so, these tests can help to control faking or other examinee’s response biases), c) behavioral tests (personality traits are measured from non-verbal responses rather than from self-reports), and d) situational tests (the examinee faces a conflictive situation and has to select the option that best describes what he or she will do). The paper evaluates these types of tests, comments on their pros and cons and provides some specific examples. Key words: Computerized adaptive test, Situational test, Behavioral test, Ipsative test and y automatic item generation. VL - 31 ER - TY - JOUR T1 - Item selection rules in computerized adaptive testing: Accuracy and security JF - Methodology Y1 - 2009 A1 - Barrada, J A1 - Olea, J. A1 - Ponsoda, V. A1 - Abad, F. J. VL - 5 N1 - (PDF file, 445 KB) ER - TY - JOUR T1 - Multiple Maximum Exposure Rates in Computerized Adaptive Testing JF - Applied Psychological Measurement Y1 - 2009 A1 - Barrada, Juan Ramón A1 - Veldkamp, Bernard P. A1 - Olea, Julio AB -

Computerized adaptive testing is subject to security problems, as the item bank content remains operative over long periods and administration time is flexible for examinees. Spreading the content of a part of the item bank could lead to an overestimation of the examinees' trait level. The most common way of reducing this risk is to impose a maximum exposure rate (rmax) that no item should exceed. Several methods have been proposed with this aim. All of these methods establish a single value of rmax throughout the test. This study presents a new method, the multiple-rmax method, that defines as many values of rmax as the number of items presented in the test. In this way, it is possible to impose a high degree of randomness in item selection at the beginning of the test, leaving the administration of items with the best psychometric properties to the moment when the trait level estimation is most accurate. The implementation of the multiple-r max method is described and is tested in simulated item banks and in an operative bank. Compared with a single maximum exposure method, the new method has a more balanced usage of the item bank and delays the possible distortion of trait estimation due to security problems, with either no or only slight decrements of measurement accuracy.

VL - 33 UR - http://apm.sagepub.com/content/33/1/58.abstract ER - TY - CHAP T1 - Test overlap rate and item exposure rate as indicators of test security in CATs Y1 - 2009 A1 - Barrada, J A1 - Olea, J. A1 - Ponsoda, V. A1 - Abad, F. J. CY - D. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing. N1 - PDF File, 261 K ER - TY - JOUR T1 - Comparability of Computer-Based and Paper-and-Pencil Testing in K–12 Reading Assessments JF - Educational and Psychological Measurement Y1 - 2008 A1 - Shudong Wang, A1 - Hong Jiao, A1 - Young, Michael J. A1 - Brooks, Thomas A1 - Olson, John AB -

In recent years, computer-based testing (CBT) has grown in popularity, is increasingly being implemented across the United States, and will likely become the primary mode for delivering tests in the future. Although CBT offers many advantages over traditional paper-and-pencil testing, assessment experts, researchers, practitioners, and users have expressed concern about the comparability of scores between the two test administration modes. To help provide an answer to this issue, a meta-analysis was conducted to synthesize the administration mode effects of CBTs and paper-and-pencil tests on K—12 student reading assessments. Findings indicate that the administration mode had no statistically significant effect on K—12 student reading achievement scores. Four moderator variables—study design, sample size, computer delivery algorithm, and computer practice—made statistically significant contributions to predicting effect size. Three moderator variables—grade level, type of test, and computer delivery method—did not affect the differences in reading scores between test modes.

VL - 68 UR - http://epm.sagepub.com/content/68/1/5.abstract ER - TY - JOUR T1 - Incorporating randomness in the Fisher information for improving item-exposure control in CATs JF - British Journal of Mathematical and Statistical Psychology Y1 - 2008 A1 - Barrada, J A1 - Olea, J. A1 - Ponsoda, V. A1 - Abad, F. J. VL - 61 ER - TY - JOUR T1 - An initial application of computerized adaptive testing (CAT) for measuring disability in patients with low back pain JF - BMC Musculoskelet Disorders Y1 - 2008 A1 - Elhan, A. H. A1 - Oztuna, D. A1 - Kutlay, S. A1 - Kucukdeveci, A. A. A1 - Tennant, A. AB - ABSTRACT: BACKGROUND: Recent approaches to outcome measurement involving Computerized Adaptive Testing (CAT) offer an approach for measuring disability in low back pain (LBP) in a way that can reduce the burden upon patient and professional. The aim of this study was to explore the potential of CAT in LBP for measuring disability as defined in the International Classification of Functioning, Disability and Health (ICF) which includes impairments, activity limitation, and participation restriction. METHODS: 266 patients with low back pain answered questions from a range of widely used questionnaires. An exploratory factor analysis (EFA) was used to identify disability dimensions which were then subjected to Rasch analysis. Reliability was tested by internal consistency and person separation index (PSI). Discriminant validity of disability levels were evaluated by Spearman correlation coefficient (r), intraclass correlation coefficient [ICC(2,1)] and the Bland-Altman approach. A CAT was developed for each dimension, and the results checked against simulated and real applications from a further 133 patients. RESULTS: Factor analytic techniques identified two dimensions named "body functions" and "activity-participation". After deletion of some items for failure to fit the Rasch model, the remaining items were mostly free of Differential Item Functioning (DIF) for age and gender. Reliability exceeded 0.90 for both dimensions. The disability levels generated using all items and those obtained from the real CAT application were highly correlated (i.e. >0.97 for both dimensions). On average, 19 and 14 items were needed to estimate the precise disability levels using the initial CAT for the first and second dimension. However, a marginal increase in the standard error of the estimate across successive iterations substantially reduced the number of items required to make an estimate. CONCLUSIONS: Using a combination approach of EFA and Rasch analysis this study has shown that it is possible to calibrate items onto a single metric in a way that can be used to provide the basis of a CAT application. Thus there is an opportunity to obtain a wide variety of information to evaluate the biopsychosocial model in its more complex forms, without necessarily increasing the burden of information collection for patients. VL - 9 SN - 1471-2474 (Electronic) N1 - Journal articleBMC musculoskeletal disordersBMC Musculoskelet Disord. 2008 Dec 18;9(1):166. ER - TY - JOUR T1 - Rotating item banks versus restriction of maximum exposure rates in computerized adaptive testing JF - Spanish Journal of Psychology Y1 - 2008 A1 - Barrada, J A1 - Olea, J. A1 - Abad, F. J. KW - *Character KW - *Databases KW - *Software Design KW - Aptitude Tests/*statistics & numerical data KW - Bias (Epidemiology) KW - Computing Methodologies KW - Diagnosis, Computer-Assisted/*statistics & numerical data KW - Educational Measurement/*statistics & numerical data KW - Humans KW - Mathematical Computing KW - Psychometrics/statistics & numerical data AB -

If examinees were to know, beforehand, part of the content of a computerized adaptive test, their estimated trait levels would then have a marked positive bias. One of the strategies to avoid this consists of dividing a large item bank into several sub-banks and rotating the sub-bank employed (Ariel, Veldkamp & van der Linden, 2004). This strategy permits substantial improvements in exposure control at little cost to measurement accuracy, However, we do not know whether this option provides better results than using the master bank with greater restriction in the maximum exposure rates (Sympson & Hetter, 1985). In order to investigate this issue, we worked with several simulated banks of 2100 items, comparing them, for RMSE and overlap rate, with the same banks divided in two, three... up to seven sub-banks. By means of extensive manipulation of the maximum exposure rate in each bank, we found that the option of rotating banks slightly outperformed the option of restricting maximum exposure rate of the master bank by means of the Sympson-Hetter method.

VL - 11 SN - 1138-7416 N1 - Barrada, Juan RamonOlea, JulioAbad, Francisco JoseResearch Support, Non-U.S. Gov'tSpainThe Spanish journal of psychologySpan J Psychol. 2008 Nov;11(2):618-25. ER - TY - JOUR T1 - IRT health outcomes data analysis project: an overview and summary JF - Quality of Life Research Y1 - 2007 A1 - Cook, K. F. A1 - Teal, C. R. A1 - Bjorner, J. B. A1 - Cella, D. A1 - Chang, C-H. A1 - Crane, P. K. A1 - Gibbons, L. E. A1 - Hays, R. D. A1 - McHorney, C. A. A1 - Ocepek-Welikson, K. A1 - Raczek, A. E. A1 - Teresi, J. A. A1 - Reeve, B. B. KW - *Data Interpretation, Statistical KW - *Health Status KW - *Quality of Life KW - *Questionnaires KW - *Software KW - Female KW - HIV Infections/psychology KW - Humans KW - Male KW - Neoplasms/psychology KW - Outcome Assessment (Health Care)/*methods KW - Psychometrics KW - Stress, Psychological AB - BACKGROUND: In June 2004, the National Cancer Institute and the Drug Information Association co-sponsored the conference, "Improving the Measurement of Health Outcomes through the Applications of Item Response Theory (IRT) Modeling: Exploration of Item Banks and Computer-Adaptive Assessment." A component of the conference was presentation of a psychometric and content analysis of a secondary dataset. OBJECTIVES: A thorough psychometric and content analysis was conducted of two primary domains within a cancer health-related quality of life (HRQOL) dataset. RESEARCH DESIGN: HRQOL scales were evaluated using factor analysis for categorical data, IRT modeling, and differential item functioning analyses. In addition, computerized adaptive administration of HRQOL item banks was simulated, and various IRT models were applied and compared. SUBJECTS: The original data were collected as part of the NCI-funded Quality of Life Evaluation in Oncology (Q-Score) Project. A total of 1,714 patients with cancer or HIV/AIDS were recruited from 5 clinical sites. MEASURES: Items from 4 HRQOL instruments were evaluated: Cancer Rehabilitation Evaluation System-Short Form, European Organization for Research and Treatment of Cancer Quality of Life Questionnaire, Functional Assessment of Cancer Therapy and Medical Outcomes Study Short-Form Health Survey. RESULTS AND CONCLUSIONS: Four lessons learned from the project are discussed: the importance of good developmental item banks, the ambiguity of model fit results, the limits of our knowledge regarding the practical implications of model misfit, and the importance in the measurement of HRQOL of construct definition. With respect to these lessons, areas for future research are suggested. The feasibility of developing item banks for broad definitions of health is discussed. VL - 16 SN - 0962-9343 (Print) N1 - Cook, Karon FTeal, Cayla RBjorner, Jakob BCella, DavidChang, Chih-HungCrane, Paul KGibbons, Laura EHays, Ron DMcHorney, Colleen AOcepek-Welikson, KatjaRaczek, Anastasia ETeresi, Jeanne AReeve, Bryce B1U01AR52171-01/AR/United States NIAMSR01 (CA60068)/CA/United States NCIY1-PC-3028-01/PC/United States NCIResearch Support, N.I.H., ExtramuralNetherlandsQuality of life research : an international journal of quality of life aspects of treatment, care and rehabilitationQual Life Res. 2007;16 Suppl 1:121-32. Epub 2007 Mar 10. ER - TY - JOUR T1 - Methods for restricting maximum exposure rate in computerized adaptative testing JF - Methodology: European Journal of Research Methods for the Behavioral and Social Sciences Y1 - 2007 A1 - Barrada, J A1 - Olea, J. A1 - Ponsoda, V. KW - computerized adaptive testing KW - item bank security KW - item exposure control KW - overlap rate KW - Sympson-Hetter method AB - The Sympson-Hetter (1985) method provides a means of controlling maximum exposure rate of items in Computerized Adaptive Testing. Through a series of simulations, control parameters are set that mark the probability of administration of an item on being selected. This method presents two main problems: it requires a long computation time for calculating the parameters and the maximum exposure rate is slightly above the fixed limit. Van der Linden (2003) presented two alternatives which appear to solve both of the problems. The impact of these methods in the measurement accuracy has not been tested yet. We show how these methods over-restrict the exposure of some highly discriminating items and, thus, the accuracy is decreased. It also shown that, when the desired maximum exposure rate is near the minimum possible value, these methods offer an empirical maximum exposure rate clearly above the goal. A new method, based on the initial estimation of the probability of administration and the probability of selection of the items with the restricted method (Revuelta & Ponsoda, 1998), is presented in this paper. It can be used with the Sympson-Hetter method and with the two van der Linden's methods. This option, when used with Sympson-Hetter, speeds the convergence of the control parameters without decreasing the accuracy. (PsycINFO Database Record (c) 2007 APA, all rights reserved) PB - Hogrefe & Huber Publishers GmbH: Germany VL - 3 SN - 1614-1881 (Print); 1614-2241 (Electronic) ER - TY - CHAP T1 - Up-and-down procedures for approximating optimal designs using person-response functions Y1 - 2007 A1 - Sheng, Y. A1 - Flournoy, N. A1 - Osterlind, S. J. CY - D. J. Weiss (Ed.). Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing. N1 - {PDF file, 1,042 KB} ER - TY - CHAP T1 - Validity and decision issues in selecting a CAT measurement model Y1 - 2007 A1 - Olsen, J. B. A1 - Bunderson, C. V CY - D. J. Weiss (Ed.). Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing. N1 - {PDF file, 977 KB} ER - TY - JOUR T1 - Maximum information stratification method for controlling item exposure in computerized adaptive testing JF - Psicothema Y1 - 2006 A1 - Barrada, J A1 - Mazuela, P. A1 - Olea, J. KW - *Artificial Intelligence KW - *Microcomputers KW - *Psychological Tests KW - *Software Design KW - Algorithms KW - Chi-Square Distribution KW - Humans KW - Likelihood Functions AB - The proposal for increasing the security in Computerized Adaptive Tests that has received most attention in recent years is the a-stratified method (AS - Chang and Ying, 1999): at the beginning of the test only items with low discrimination parameters (a) can be administered, with the values of the a parameters increasing as the test goes on. With this method, distribution of the exposure rates of the items is less skewed, while efficiency is maintained in trait-level estimation. The pseudo-guessing parameter (c), present in the three-parameter logistic model, is considered irrelevant, and is not used in the AS method. The Maximum Information Stratified (MIS) model incorporates the c parameter in the stratification of the bank and in the item-selection rule, improving accuracy by comparison with the AS, for item banks with a and b parameters correlated and uncorrelated. For both kinds of banks, the blocking b methods (Chang, Qian and Ying, 2001) improve the security of the item bank.Método de estratificación por máxima información para el control de la exposición en tests adaptativos informatizados. La propuesta para aumentar la seguridad en los tests adaptativos informatizados que ha recibido más atención en los últimos años ha sido el método a-estratificado (AE - Chang y Ying, 1999): en los momentos iniciales del test sólo pueden administrarse ítems con bajos parámetros de discriminación (a), incrementándose los valores del parámetro a admisibles según avanza el test. Con este método la distribución de las tasas de exposición de los ítems es más equilibrada, manteniendo una adecuada precisión en la medida. El parámetro de pseudoadivinación (c), presente en el modelo logístico de tres parámetros, se supone irrelevante y no se incorpora en el AE. El método de Estratificación por Máxima Información (EMI) incorpora el parámetro c a la estratificación del banco y a la regla de selección de ítems, mejorando la precisión en comparación con AE, tanto para bancos donde los parámetros a y b correlacionan como para bancos donde no. Para ambos tipos de bancos, los métodos de bloqueo de b (Chang, Qian y Ying, 2001) mejoran la seguridad del banco. VL - 18 SN - 0214-9915 (Print) N1 - Barrada, Juan RamonMazuela, PalomaOlea, JulioResearch Support, Non-U.S. Gov'tSpainPsicothemaPsicothema. 2006 Feb;18(1):156-9. ER - TY - CONF T1 - Multiple maximum exposure rates in computerized adaptive testing T2 - Paper presented at the SMABS-EAM Conference Y1 - 2006 A1 - Barrada, J A1 - Veldkamp, B. P. A1 - Olea, J. JF - Paper presented at the SMABS-EAM Conference CY - Budapest, Hungary ER - TY - JOUR T1 - Dynamic assessment of health outcomes: Time to let the CAT out of the bag? JF - Health Services Research Y1 - 2005 A1 - Cook, K. F. A1 - O'Malley, K. J. A1 - Roddey, T. S. KW - computer adaptive testing KW - Item Response Theory KW - self reported health outcomes AB - Background: The use of item response theory (IRT) to measure self-reported outcomes has burgeoned in recent years. Perhaps the most important application of IRT is computer-adaptive testing (CAT), a measurement approach in which the selection of items is tailored for each respondent. Objective. To provide an introduction to the use of CAT in the measurement of health outcomes, describe several IRT models that can be used as the basis of CAT, and discuss practical issues associated with the use of adaptive scaling in research settings. Principal Points: The development of a CAT requires several steps that are not required in the development of a traditional measure including identification of "starting" and "stopping" rules. CAT's most attractive advantage is its efficiency. Greater measurement precision can be achieved with fewer items. Disadvantages of CAT include the high cost and level of technical expertise required to develop a CAT. Conclusions: Researchers, clinicians, and patients benefit from the availability of psychometrically rigorous measures that are not burdensome. CAT outcome measures hold substantial promise in this regard, but their development is not without challenges. (PsycINFO Database Record (c) 2007 APA, all rights reserved) PB - Blackwell Publishing: United Kingdom VL - 40 SN - 0017-9124 (Print); 1475-6773 (Electronic) ER - TY - JOUR T1 - Assisted self-adapted testing: A comparative study JF - European Journal of Psychological Assessment Y1 - 2004 A1 - Hontangas, P. A1 - Olea, J. A1 - Ponsoda, V. A1 - Revuelta, J. A1 - Wise, S. L. KW - Adaptive Testing KW - Anxiety KW - Computer Assisted Testing KW - Psychometrics KW - Test AB - A new type of self-adapted test (S-AT), called Assisted Self-Adapted Test (AS-AT), is presented. It differs from an ordinary S-AT in that prior to selecting the difficulty category, the computer advises examinees on their best difficulty category choice, based on their previous performance. Three tests (computerized adaptive test, AS-AT, and S-AT) were compared regarding both their psychometric (precision and efficiency) and psychological (anxiety) characteristics. Tests were applied in an actual assessment situation, in which test scores determined 20% of term grades. A sample of 173 high school students participated. Neither differences in posttest anxiety nor ability were obtained. Concerning precision, AS-AT was as precise as CAT, and both revealed more precision than S-AT. It was concluded that AS-AT acted as a CAT concerning precision. Some hints, but not conclusive support, of the psychological similarity between AS-AT and S-AT was also found. (PsycINFO Database Record (c) 2005 APA ) (journal abstract) VL - 20 ER - TY - CHAP T1 - A Learning Environment for English for Academic Purposes Based on Adaptive Tests and Task-Based Systems T2 - Intelligent Tutoring Systems Y1 - 2004 A1 - Gonçalves, Jean P. A1 - Aluisio, Sandra M. A1 - de Oliveira, Leandro H.M. A1 - Oliveira Jr., Osvaldo N. ED - Lester, James C. ED - Vicari, Rosa Maria ED - Paraguaçu, Fábio JF - Intelligent Tutoring Systems T3 - Lecture Notes in Computer Science PB - Springer Berlin / Heidelberg VL - 3220 SN - 978-3-540-22948-3 UR - http://dx.doi.org/10.1007/978-3-540-30139-4_1 ER - TY - CHAP T1 - Computerized adaptive testing Y1 - 2003 A1 - Ponsoda, V. A1 - Olea, J. CY - R. Fernández-Ballesteros (Ed.): Encyclopaedia of Psychological Assessment. London: Sage. ER - TY - JOUR T1 - Development and psychometric evaluation of the Flexilevel Scale of Shoulder Function (FLEX-SF) JF - Medical Care (in press) Y1 - 2003 A1 - Cook, K. F. A1 - Roddey, T. S. A1 - Gartsman, G M A1 - Olson, S L N1 - #CO03-01 ER - TY - CHAP T1 - Item selection in polytomous CAT T2 - New developments in psychometrics Y1 - 2003 A1 - Veldkamp, B. P. ED - A. Okada ED - K. Shigenasu ED - Y. Kano ED - J. Meulman KW - computerized adaptive testing JF - New developments in psychometrics PB - Psychometric Society, Springer CY - Tokyo, Japan ER - TY - JOUR T1 - Psychometric and psychological effects of item selection and review on computerized testing JF - Educational and Psychological Measurement Y1 - 2003 A1 - Revuelta, J. A1 - Ximénez, M. C. A1 - Olea, J. VL - 63 ER - TY - JOUR T1 - Psychometric and Psychological Effects of Item Selection and Review on Computerized Testing JF - Educational and Psychological Measurement Y1 - 2003 A1 - Revuelta, Javier A1 - Ximénez, M. Carmen A1 - Olea, Julio AB -

Psychometric properties of computerized testing, together with anxiety and comfort of examinees, are investigated in relation to item selection routine and the opportunity for response review. Two different hypotheses involving examinee anxiety were used to design test properties: perceived control and perceived performance. The study involved three types of administration of a computerized English test for Spanish speakers (adaptive, easy adaptive, and fixed) and four review conditions (no review, review at end, review by blocks of 5 items, and review item-by-item). These were applied to a sample of 557 first-year psychology undergraduate students to examine main and interaction effects of test type and review on psychometric and psychological variables. Statistically significant effects were found in test precision among the different types of test. Response review improved ability estimates and increased testing time. No psychological effects on anxiety were found. Examinees in all review conditions considered more important the possibility of review than those who were not allowed to review. These results concur with previous findings on examinees' preference for item review and raise some issues that should be addressed in the field of tests with item review.

VL - 63 UR - http://epm.sagepub.com/content/63/5/791.abstract ER - TY - ABST T1 - Tests adaptativos informatizados (Computerized adaptive testing) Y1 - 2003 A1 - Olea, J. A1 - Ponsoda, V. CY - Madrid: UNED Ediciones N1 - [In Spanish] ER - TY - JOUR T1 - Self-adapted testing: An overview JF - International Journal of Continuing Engineering Education and Life-Long Learning Y1 - 2002 A1 - Wise, S. L. A1 - Ponsoda, V. A1 - Olea, J. VL - 12 ER - TY - JOUR T1 - Technology solutions for testing JF - School Administrator Y1 - 2002 A1 - Olson, A. AB - Northwest Evaluation Association in Portland, Oregon, consults with state and local educators on assessment issues. Describes several approaches in place at school districts that are using some combination of computer-based tests to measure student growth. The computerized adaptive test adjusts items based on a student's answer in "real time." On-demand testing provides almost instant scoring. (MLF) VL - 4 ER - TY - JOUR T1 - Development of an adaptive multimedia program to collect patient health data JF - American Journal of Preventative Medicine Y1 - 2001 A1 - Sutherland, L. A. A1 - Campbell, M. A1 - Ornstein, K. A1 - Wildemuth, B. A1 - Lobach, D. VL - 21 ER - TY - JOUR T1 - Requerimientos, aplicaciones e investigación en tests adaptativos informatizados [Requirements, applications, and investigation in computerized adaptive testing] JF - Apuntes de Psicologia Y1 - 2001 A1 - Olea Díaz, J. A1 - Ponsoda Gil, V. A1 - Revuelta Menéndez, J. A1 - Hontangas Beltrán, P. A1 - Abad, F. J. KW - Computer Assisted Testing KW - English as Second Language KW - Psychometrics computerized adaptive testing AB - Summarizes the main requirements and applications of computerized adaptive testing (CAT) with emphasis on the differences between CAT and conventional computerized tests. Psychometric properties of estimations based on CAT, item selection strategies, and implementation software are described. Results of CAT studies in Spanish-speaking samples are described. Implications for developing a CAT measuring the English vocabulary of Spanish-speaking students are discussed. (PsycINFO Database Record (c) 2005 APA ) VL - 19 ER - TY - JOUR T1 - The choice of item difficulty in self adapted testing JF - European Journal of Psychological Assessment Y1 - 2000 A1 - Hontangas, P. A1 - Ponsoda, V. A1 - Olea, J. A1 - Wise, S. L. VL - 16 IS - 1 ER - TY - JOUR T1 - The impact of receiving the same items on consecutive computer adaptive test administrations JF - Journal of Applied Measurement Y1 - 2000 A1 - O'Neill, T. A1 - Lunz, M. E. A1 - Thiede, K. AB - Addresses item exposure in a Computerized Adaptive Test (CAT) when the item selection algorithm is permitted to present examinees with questions that they have already been asked in a previous test administration. The data were from a national certification exam in medical technology. The responses of 178 repeat examinees were compared. The results indicate that the combined use of an adaptive algorithm to select items and latent trait theory to estimate person ability provides substantial protection from score contamination. The implications for constraints that prohibit examinees from seeing an item twice are discussed. (PsycINFO Database Record (c) 2002 APA, all rights reserved). VL - 1 N1 - Richard M Smith, US ER - TY - JOUR T1 - Los tests adaptativos informatizados en la frontera del siglo XXI: Una revisión [Computerized adaptive tests at the turn of the 21st century: A review] JF - Metodología de las Ciencias del Comportamiento Y1 - 2000 A1 - Hontangas, P. A1 - Ponsoda, V. A1 - Olea, J. A1 - Abad, F. J. KW - computerized adaptive testing VL - 2 SN - 1575-9105 ER - TY - JOUR T1 - Psychometric and psychological effects of review on computerized fixed and adaptive tests JF - Psicolgia Y1 - 2000 A1 - Olea, J. A1 - Revuelta, J. A1 - Ximenez, M. C. A1 - Abad, F. J. VL - 21 ER - TY - CHAP T1 - The development of a computerized adaptive selection system for computer programmers in a financial services company Y1 - 1999 A1 - Zickar, M.. J. A1 - Overton, R. C. A1 - Taylor, L. R. A1 - Harms, H. J. CY - F. Drasgow and J. B. Olsen (Eds.), Innvoations in computerized assessment (p. 7-33). Mahwah NJ Erlbaum. ER - TY - JOUR T1 - The effects of test difficulty manipulation in computerized adaptive testing and self-adapted testing JF - Applied Measurement in Education Y1 - 1999 A1 - Ponsoda, V. A1 - Olea, J. A1 - Rodriguez, M. S. A1 - Revuelta, J. VL - 12 ER - TY - BOOK T1 - Innovations in computerized assessment Y1 - 1999 A1 - F Drasgow A1 - Olson-Buchanan, J. B. KW - computerized adaptive testing AB - Chapters in this book present the challenges and dilemmas faced by researchers as they created new computerized assessments, focusing on issues addressed in developing, scoring, and administering the assessments. Chapters are: (1) "Beyond Bells and Whistles; An Introduction to Computerized Assessment" (Julie B. Olson-Buchanan and Fritz Drasgow); (2) "The Development of a Computerized Selection System for Computer Programmers in a Financial Services Company" (Michael J. Zickar, Randall C. Overton, L. Rogers Taylor, and Harvey J. Harms); (3) "Development of the Computerized Adaptive Testing Version of the Armed Services Vocational Aptitude Battery" (Daniel O. Segall and Kathleen E. Moreno); (4) "CAT for Certification and Licensure" (Betty A. Bergstrom and Mary E. Lunz); (5) "Developing Computerized Adaptive Tests for School Children" (G. Gage Kingsbury and Ronald L. Houser); (6) "Development and Introduction of a Computer Adaptive Graduate Record Examinations General Test" (Craig N. Mills); (7) "Computer Assessment Using Visual Stimuli: A Test of Dermatological Skin Disorders" (Terry A. Ackerman, John Evans, Kwang-Seon Park, Claudia Tamassia, and Ronna Turner); (8) "Creating Computerized Adaptive Tests of Music Aptitude: Problems, Solutions, and Future Directions" (Walter P. Vispoel); (9) "Development of an Interactive Video Assessment: Trials and Tribulations" (Fritz Drasgow, Julie B. Olson-Buchanan, and Philip J. Moberg); (10) "Computerized Assessment of Skill for a Highly Technical Job" (Mary Ann Hanson, Walter C. Borman, Henry J. Mogilka, Carol Manning, and Jerry W. Hedge); (11) "Easing the Implementation of Behavioral Testing through Computerization" (Wayne A. Burroughs, Janet Murray, S. Scott Wesley, Debra R. Medina, Stacy L. Penn, Steven R. Gordon, and Michael Catello); and (12) "Blood, Sweat, and Tears: Some Final Comments on Computerized Assessment." (Fritz Drasgow and Julie B. Olson-Buchanan). Each chapter contains references. (Contains 17 tables and 21 figures.) (SLD) PB - Lawrence Erlbaum Associates, Inc. CY - Mahwah, N.J. N1 - EDRS Availability: None. Lawrence Erlbaum Associates, Inc., Publishers, 10 Industrial Avenue, Mahwah, New Jersey 07430-2262 (paperback: ISBN-0-8058-2877-X, $29.95; clothbound: ISBN-0-8058-2876-1, $59.95). Tel: 800-926-6579 (Toll Free). ER - TY - ABST T1 - Tests informatizados: Fundamentos y aplicaciones (Computerized testing: Fundamentals and applications Y1 - 1999 A1 - Olea, J. A1 - Ponsoda, V. A1 - Prieto, G., Eds. CY - Madrid: Pirmide. N1 - [In Spanish] ER - TY - JOUR T1 - Adapting to adaptive testing JF - Personnel Psychology Y1 - 1997 A1 - Overton, R. C. A1 - Harms, H. J. A1 - Taylor, L. R. A1 - Zickar, M.. J. VL - 50 ER - TY - JOUR T1 - An investigation of self-adapted testing in a Spanish high school population JF - Educational and Psychological Measurement Y1 - 1997 A1 - Ponsoda, V. A1 - Wise, S. L. A1 - Olea, J. A1 - Revuelta, J. VL - 57 ER - TY - JOUR T1 - Propiedades psicometricas du un test adaptivo informatizado do vocabulario ingles [Psychometric properties of a computerized adaptive tests for the measurement of English vocabulary] JF - Estudios de Psicologica Y1 - 1996 A1 - Olea., J. A1 - Ponsoda, V. A1 - Revuelta, J. A1 - Belchi, J. VL - 55 N1 - [In Spanish] ER - TY - CHAP T1 - Test adaptativos informatizados [Computerized adaptive testing] T2 - Psicometría Y1 - 1996 A1 - Olea, J. A1 - Ponsoda, V. JF - Psicometría PB - Universitas CY - Madrid, UNED ER - TY - JOUR T1 - Effects and underlying mechanisms of self-adapted testing JF - Journal of Educational Psychology Y1 - 1995 A1 - Rocklin, T. R. A1 - O’Donnell, A. M. A1 - Holst, P. M. VL - 87 ER - TY - CONF T1 - Tests adaptivos y autoadaptados informatizados: Effects en la ansiedad y en la pecision de las estimaciones [SATs and CATS: Effects on enxiety and estimate precision] T2 - Paper presented at the Fourth Symposium de Metodologia de las Ciencies del Comportamiento Y1 - 1995 A1 - Olea, J. A1 - Ponsoda, V. A1 - Wise, S. L. JF - Paper presented at the Fourth Symposium de Metodologia de las Ciencies del Comportamiento CY - Murcia, Spain ER - TY - JOUR T1 - ADTEST: A computer-adaptive tests based on the maximum information principle JF - Educational and Psychological Measurement Y1 - 1994 A1 - Ponsoda, V. A1 - Olea, J., A1 - Revuelta, J. VL - 54 ER - TY - ABST T1 - Computerized Testing (Research Report 94-22). Y1 - 1994 A1 - Oltman, P. K. CY - Princeton NJ: Educational Testing Service ER - TY - CONF T1 - Effects of feedback during self-adapted testing on estimates of ability T2 - Paper presented at the annual meeting of the American Educational Research Association Y1 - 1992 A1 - Holst, P. M. A1 - O’Donnell, A. M. A1 - Rocklin, T. R. JF - Paper presented at the annual meeting of the American Educational Research Association CY - San Francisco ER - TY - CONF T1 - An empirical comparison of self-adapted and maximum information item selection T2 - Paper presented at the annual meeting of the American Educational Research Association Y1 - 1991 A1 - Rocklin, T. R. A1 - O’Donnell, A. M. JF - Paper presented at the annual meeting of the American Educational Research Association CY - Chicago ER - TY - JOUR T1 - Applying computerized adaptive testing in schools JF - Measurement and Evaluation in Counseling and Development Y1 - 1990 A1 - Olson, J. B VL - 23 ER - TY - JOUR T1 - Comparisons of paper-administered, computer-administered and computerized adaptive achievement tests JF - Journal of Educational Computing Research Y1 - 1989 A1 - Olson, J. B A1 - Maynes, D. D. A1 - Slawson, D. A1 - Ho, K AB - This research study was designed to compare student achievement scores from three different testing methods: paper-administered testing, computer-administered testing, and computerized adaptive testing. The three testing formats were developed from the California Assessment Program (CAP) item banks for grades three and six. The paper-administered and the computer-administered tests were identical in item content, format, and sequence. The computerized adaptive test was a tailored or adaptive sequence of the items in the computer-administered test. VL - 5 ER - TY - ABST T1 - The four generations of computerized educational measurement (Research Report 98-35) Y1 - 1988 A1 - Bunderson, C. V A1 - Inouye, D. K A1 - Olsen, J. B. CY - Princeton NJ: Educational Testing Service. ER - TY - JOUR T1 - Self-adapted testing: A performance improving variation of computerized adaptive testing JF - Journal of Educational Psychology Y1 - 1987 A1 - Rocklin, T. R., A1 - O’Donnell, A. M. VL - 79 ER - TY - CONF T1 - Comparison and equating of paper-administered, computer-administered, and computerized adaptive tests of achievement T2 - Paper presented at the annual meeting of the American Educational Research Association Y1 - 1986 A1 - Olsen, J. B. A1 - Maynes, D. D. A1 - Slawson, D. A1 - Ho, K JF - Paper presented at the annual meeting of the American Educational Research Association CY - San Francisco CA ER - TY - CHAP T1 - The four generations of computerized educational measurement Y1 - 1986 A1 - Bunderson, C. V A1 - Inouye, D. K A1 - Olsen, J. B. CY - In R. L. Linn (Ed.), Educational Measurement (3rd ed and pp. 367-407). New York: Macmillan. ER - TY - JOUR T1 - Ability measurement, test bias reduction, and psychological reactions to testing as a function of computer adaptive testing versus conventional testing JF - Dissertation Abstracts International Y1 - 1982 A1 - Orban, J. A. KW - computerized adaptive testing VL - 42 ER - TY - BOOK T1 - Ability measurement, test bias reduction, and psychological reactions to testing as a function of computer adaptive testing versus conventional Y1 - 1981 A1 - Orban, J. A. CY - Unpublished doctoral dissertation, Virginia Polytechnic Institute and State University. Dissertational Abstracts International, 1982, 42,(10-B), 4233 N1 - #OR81-01 ER - TY - ABST T1 - Bayesian sequential design and analysis of dichotomous experiments with special reference to mental testing Y1 - 1979 A1 - Owen, R. J. CY - Princeton NJ: Educational Testing Service ER - TY - JOUR T1 - A Bayesian sequential procedure for quantal response in the context of adaptive mental testing JF - Journal of the American Statistical Association Y1 - 1975 A1 - Owen, R. J. VL - 70 ER - TY - THES T1 - An evaluation of the self-scoring flexilevel testing model Y1 - 1974 A1 - Olivier, P. PB - Florida State University ER - TY - BOOK T1 - An evaluation of the self-scoring flexilevel testing model Y1 - 1974 A1 - Olivier, P. CY - Unpublished dissertation, Florida State University. Dissertation Abstracts International, 35 (7-A), 4257 ER - TY - ABST T1 - An overview of tailored testing (unpublished manuscript) Y1 - 1973 A1 - Olivier, P. CY - Florida State University, Program of Educational Evaluation and Research Design ER - TY - ABST T1 - A Bayesian approach to tailored testing (Research Report 69-92) Y1 - 1969 A1 - Owen, R. J. CY - Princeton NJ: Educational Testing Service ER -