03045nas a2200241 4500008004100000020004100041245017000082210007100252250001500323300001000338490000700348520214300355653001402498653006602512653001102578653001302589100001402602700001202616700001402628700001502642700001702657856012902674 2010 spa d a0214-9915 (Print)0214-9915 (Linking)00aDeterioro de parámetros de los ítems en tests adaptativos informatizados: estudio con eCAT [Item parameter drift in computerized adaptive testing: Study with eCAT]0 aDeterioro de parámetros de los ítems en tests adaptativos inform a2010/04/29 a340-70 v223 a
En el presente trabajo se muestra el análisis realizado sobre un Test Adaptativo Informatizado (TAI) diseñado para la evaluación del nivel de inglés, denominado eCAT, con el objetivo de estudiar el deterioro de parámetros (parameter drift) producido desde la calibración inicial del banco de ítems. Se ha comparado la calibración original desarrollada para la puesta en servicio del TAI (N= 3224) y la calibración actual obtenida con las aplicaciones reales del TAI (N= 7254). Se ha analizado el Funcionamiento Diferencial de los Ítems (FDI) en función de los parámetros utilizados y se ha simulado el impacto que sobre el nivel de rasgo estimado tiene la variación en los parámetros. Los resultados muestran que se produce especialmente un deterioro de los parámetros a y c, que hay unimportante número de ítems del banco para los que existe FDI y que la variación de los parámetros produce un impacto moderado en la estimación de θ de los evaluados con nivel de inglés alto. Se concluye que los parámetros de los ítems se han deteriorado y deben ser actualizados.Item parameter drift in computerized adaptive testing: Study with eCAT. This study describes the parameter drift analysis conducted on eCAT (a Computerized Adaptive Test to assess the written English level of Spanish speakers). The original calibration of the item bank (N = 3224) was compared to a new calibration obtained from the data provided by most eCAT operative administrations (N =7254). A Differential Item Functioning (DIF) study was conducted between the original and the new calibrations. The impact that the new parameters have on the trait level estimates was obtained by simulation. Results show that parameter drift is found especially for a and c parameters, an important number of bank items show DIF, and the parameter change has a moderate impact on high-level-English θ estimates. It is then recommended to replace the original estimates by the new set. by the new set.
10a*Software10aEducational Measurement/*methods/*statistics & numerical data10aHumans10aLanguage1 aAbad, F J1 aOlea, J1 aAguado, D1 aPonsoda, V1 aBarrada, J R uhttp://iacat.org/content/deterioro-de-par%C3%A1metros-de-los-%C3%ADtems-en-tests-adaptativos-informatizados-estudio-con-ecat02922nas a2200145 4500008004100000245009900041210006900140300001100209490000700220520238700227100001202614700001402626700001702640856011902657 2010 eng d00aTests informatizados y otros nuevos tipos de tests [Computerized and other new types of tests]0 aTests informatizados y otros nuevos tipos de tests Computerized a94-1070 v313 aRecientemente se ha producido un considerable desarrollo de los tests adaptativos informatizados, en los que el test se adapta progresivamente al rendimiento del evaluando, y de otros tipos de tests: a) los test basados en modelos (se dispone de un modelo o teoría de cómo se responde a cada ítem, lo que permite predecir su dificultad), b) los tests ipsativos (el evaluado ha de elegir entre opciones que tienen parecida deseabilidad social, por lo que pueden resultar eficaces para controlar algunos sesgos de respuestas), c) los tests conductuales (miden rasgos que ordinariamente se han venido midiendo con autoinformes, mediante tareas que requieren respuestas no verbales) y d) los tests situacionales (en los que se presenta al evaluado una situación de conflicto laboral, por ejemplo, con varias posibles soluciones, y ha de elegir la que le parece la mejor descripción de lo que el haría en esa situación). El artículo comenta las características, ventajas e inconvenientes de todos ellos y muestra algunos ejemplos de tests concretos. Palabras clave: Test adaptativo informatizado, Test situacional, Test comportamental, Test ipsativo y generación automática de ítems.The paper provides a short description of some test types that are earning considerable interest in both research and applied areas. The main feature of a computerized adaptive test is that in despite of the examinees receiving different sets of items, their test scores are in the same metric and can be directly compared. Four other test types are considered: a) model-based tests (a model or theory is available to explain the item response process and this makes the prediction of item difficulties possible), b) ipsative tests (the examinee has to select one among two or more options with similar social desirability; so, these tests can help to control faking or other examinee’s response biases), c) behavioral tests (personality traits are measured from non-verbal responses rather than from self-reports), and d) situational tests (the examinee faces a conflictive situation and has to select the option that best describes what he or she will do). The paper evaluates these types of tests, comments on their pros and cons and provides some specific examples. Key words: Computerized adaptive test, Situational test, Behavioral test, Ipsative test and y automatic item generation.1 aOlea, J1 aAbad, F J1 aBarrada, J R uhttp://iacat.org/content/tests-informatizados-y-otros-nuevos-tipos-de-tests-computerized-and-other-new-types-tests02945nas a2200205 4500008004100000020004100041245009800082210006900180250001500249260000800264300001200272490000700284520217900291653004302470653006402513100001702577700001402594700001802608856011302626 2009 eng d a0214-9915 (Print)0214-9915 (Linking)00aComparison of methods for controlling maximum exposure rates in computerized adaptive testing0 aComparison of methods for controlling maximum exposure rates in a2009/05/01 cMay a313-3200 v213 aThis paper has two objectives: (a) to provide a clear description of three methods for controlling the maximum exposure rate in computerized adaptive testing —the Symson-Hetter method, the restricted method, and the item-eligibility method— showing how all three can be interpreted as methods for constructing the variable sub-bank of items from which each examinee receives the items in his or her test; (b) to indicate the theoretical and empirical limitations of each method and to compare their performance. With the three methods, we obtained basically indistinguishable results in overlap rate and RMSE (differences in the third decimal place). The restricted method is the best method for controlling exposure rate, followed by the item-eligibility method. The worst method is the Sympson-Hetter method. The restricted method presents problems of sequential overlap rate. Our advice is to use the item-eligibility method, as it saves time and satisfies the goals of restricting maximum exposure. Comparación de métodos para el control de tasa máxima en tests adaptativos informatizados. Este artículo tiene dos objetivos: (a) ofrecer una descripción clara de tres métodos para el control de la tasa máxima en tests adaptativos informatizados, el método Symson-Hetter, el método restringido y el métodode elegibilidad del ítem, mostrando cómo todos ellos pueden interpretarse como métodos para la construcción del subbanco de ítems variable, del cual cada examinado recibe los ítems de su test; (b) señalar las limitaciones teóricas y empíricas de cada método y comparar sus resultados. Se obtienen resultados básicamente indistinguibles en tasa de solapamiento y RMSE con los tres métodos (diferencias en la tercera posición decimal). El método restringido es el mejor en el control de la tasa de exposición,seguido por el método de elegibilidad del ítem. El peor es el método Sympson-Hetter. El método restringido presenta un problema de solapamiento secuencial. Nuestra recomendación sería utilizar el método de elegibilidad del ítem, puesto que ahorra tiempo y satisface los objetivos de limitar la tasa máxima de exposición.10a*Numerical Analysis, Computer-Assisted10aPsychological Tests/*standards/*statistics & numerical data1 aBarrada, J R1 aAbad, F J1 aVeldkamp, B P uhttp://iacat.org/content/comparison-methods-controlling-maximum-exposure-rates-computerized-adaptive-testing00512nas a2200145 4500008004100000245008100041210006900122300000900191490000600200100001700206700001200223700001500235700001400250856010200264 2009 eng d00aItem selection rules in computerized adaptive testing: Accuracy and security0 aItem selection rules in computerized adaptive testing Accuracy a a7-170 v51 aBarrada, J R1 aOlea, J1 aPonsoda, V1 aAbad, F J uhttp://iacat.org/content/item-selection-rules-computerized-adaptive-testing-accuracy-and-security00583nas a2200133 4500008004100000245008400041210006900125260009700194100001700291700001200308700001500320700001400335856010000349 2009 eng d00aTest overlap rate and item exposure rate as indicators of test security in CATs0 aTest overlap rate and item exposure rate as indicators of test s aD. J. Weiss (Ed.), Proceedings of the 2009 GMAC Conference on Computerized Adaptive Testing.1 aBarrada, J R1 aOlea, J1 aPonsoda, V1 aAbad, F J uhttp://iacat.org/content/test-overlap-rate-and-item-exposure-rate-indicators-test-security-cats00542nas a2200145 4500008004100000245009900041210006900140300001200209490000700221100001700228700001200245700001500257700001400272856011000286 2008 eng d00aIncorporating randomness in the Fisher information for improving item-exposure control in CATs0 aIncorporating randomness in the Fisher information for improving a493-5130 v611 aBarrada, J R1 aOlea, J1 aPonsoda, V1 aAbad, F J uhttp://iacat.org/content/incorporating-randomness-fisher-information-improving-item-exposure-control-cats02170nas a2200301 4500008004100000020001400041245010200055210006900157250001500226300001200241490000700253520109000260653001501350653001501365653002101380653004801401653002401449653002801473653006201501653005701563653001101620653002701631653004601658100001701704700001201721700001401733856012101747 2008 eng d a1138-741600aRotating item banks versus restriction of maximum exposure rates in computerized adaptive testing0 aRotating item banks versus restriction of maximum exposure rates a2008/11/08 a618-6250 v113 aIf examinees were to know, beforehand, part of the content of a computerized adaptive test, their estimated trait levels would then have a marked positive bias. One of the strategies to avoid this consists of dividing a large item bank into several sub-banks and rotating the sub-bank employed (Ariel, Veldkamp & van der Linden, 2004). This strategy permits substantial improvements in exposure control at little cost to measurement accuracy, However, we do not know whether this option provides better results than using the master bank with greater restriction in the maximum exposure rates (Sympson & Hetter, 1985). In order to investigate this issue, we worked with several simulated banks of 2100 items, comparing them, for RMSE and overlap rate, with the same banks divided in two, three... up to seven sub-banks. By means of extensive manipulation of the maximum exposure rate in each bank, we found that the option of rotating banks slightly outperformed the option of restricting maximum exposure rate of the master bank by means of the Sympson-Hetter method.
10a*Character10a*Databases10a*Software Design10aAptitude Tests/*statistics & numerical data10aBias (Epidemiology)10aComputing Methodologies10aDiagnosis, Computer-Assisted/*statistics & numerical data10aEducational Measurement/*statistics & numerical data10aHumans10aMathematical Computing10aPsychometrics/statistics & numerical data1 aBarrada, J R1 aOlea, J1 aAbad, F J uhttp://iacat.org/content/rotating-item-banks-versus-restriction-maximum-exposure-rates-computerized-adaptive-testing01356nas a2200205 4500008004100000245016700041210007000208300001000278490000700288520051600295653003000811653003100841653004800872100001800920700001900938700002600957700002600983700001401009856012701023 2001 eng d00aRequerimientos, aplicaciones e investigación en tests adaptativos informatizados [Requirements, applications, and investigation in computerized adaptive testing]0 aRequerimientos aplicaciones e investigación en tests adaptativos a11-280 v193 aSummarizes the main requirements and applications of computerized adaptive testing (CAT) with emphasis on the differences between CAT and conventional computerized tests. Psychometric properties of estimations based on CAT, item selection strategies, and implementation software are described. Results of CAT studies in Spanish-speaking samples are described. Implications for developing a CAT measuring the English vocabulary of Spanish-speaking students are discussed. (PsycINFO Database Record (c) 2005 APA )10aComputer Assisted Testing10aEnglish as Second Language10aPsychometrics computerized adaptive testing1 aOlea Díaz, J1 aPonsoda Gil, V1 aRevuelta Menéndez, J1 aHontangas Beltrán, P1 aAbad, F J uhttp://iacat.org/content/requerimientos-aplicaciones-e-investigaci%C3%B3n-en-tests-adaptativos-informatizados-requirements00688nas a2200169 4500008004100000020001400041245015800055210006900213300001200282490000600294653003400300100001700334700001500351700001200366700001400378856012600392 2000 eng d a1575-910500aLos tests adaptativos informatizados en la frontera del siglo XXI: Una revisión [Computerized adaptive tests at the turn of the 21st century: A review]0 aLos tests adaptativos informatizados en la frontera del siglo XX a183-2160 v210acomputerized adaptive testing1 aHontangas, P1 aPonsoda, V1 aOlea, J1 aAbad, F J uhttp://iacat.org/content/los-tests-adaptativos-informatizados-en-la-frontera-del-siglo-xxi-una-revisi%C3%B3n-computerized00545nas a2200145 4500008004500000245009400045210006900139300001200208490000700220100001200227700001600239700001700255700001400272856011300286 2000 Spandsh 00aPsychometric and psychological effects of review on computerized fixed and adaptive tests0 aPsychometric and psychological effects of review on computerized a157-1730 v211 aOlea, J1 aRevuelta, J1 aXimenez, M C1 aAbad, F J uhttp://iacat.org/content/psychometric-and-psychological-effects-review-computerized-fixed-and-adaptive-tests