Computerized Adaptive Testing With the Partial Credit Model: Estimation Procedures, Population Distributions, and Item Pool Characteristics - Gorin, Joanna S.; Dodd, Barbara G.; Fitzpatrick, Steven J.; Shieh, Yann Yann (Applied Psychological Measurement, 2005)
The primary purpose of this research is to examine the impact of estimation methods, actual latent trait distributions, and item pool characteristics on the performance of a simulated computerized adaptive testing (CAT) system. In this study, three estimation procedures are compared for accuracy of estimation: maximum likelihood estimation (MLE), expected a priori (EAP), and Warm's weighted likelihood estimation (WLE). Some research has shown that MLE and EAP perform equally well under certain conditions in polytomous CAT systems, such that they match the actual latent trait distribution. However, little research has compared these methods when prior estimates of. distributions are extremely poor. In general, it appears that MLE, EAP, and WLE procedures perform equally well when using an optimal item pool. However, the use of EAP procedures may be advantageous under nonoptimal testing conditions when the item pool is not appropriately matched to the examinees.

Computerized adaptive testing with the partial credit model: Estimation procedures, population distributions, and item pool characteristics - Gorin, J.; Dodd, B. G.; Fitzpatrick, S. J.; Shieh, Y. Y. (Applied Psychological Measurement, 2005)

Computerized adaptive testing with the partial credit model: Estimation procedures, population distributions, and item pool characteristics - Gorin, J. S. (Applied Psychological Measurement, 2005)

Computerizing statewide assessments in Minnesota: A report on the feasibility of converting the Minnesota Comprehensive Assessments to a computerized adaptive format - Peterson, K.A.; Davison. M. L.; Hjelseth, L. (Office of Educational Accountability, College of Education and Human Development, University of Minnesota, 2005)

Constraining item exposure in computerized adaptive testing with shadow tests - van der Linden, W. J.; Veldkamp, B. P. (2005)

Constructing a Computerized Adaptive Test for University Applicants With Disabilities - Moshinsky, Avital; Kazin, Cathrael (Applied Measurement in Education, 2005)

Contemporary measurement techniques for rehabilitation outcomes assessment - Jette, A. M.; Haley, S. M. (Journal of Rehabilitation Medicine, 2005)
In this article, we review the limitations of traditional rehabilitation functional outcome instruments currently in use within the rehabilitation field to assess Activity and Participation domains as defined by the International Classification of Function, Disability, and Health. These include a narrow scope of functional outcomes, data incompatibility across instruments, and the precision vs feasibility dilemma. Following this, we illustrate how contemporary measurement techniques, such as item response theory methods combined with computer adaptive testing methodology, can be applied in rehabilitation to design functional outcome instruments that are comprehensive in scope, accurate, allow for compatibility across instruments, and are sensitive to clinically important change without sacrificing their feasibility. Finally, we present some of the pressing challenges that need to be overcome to provide effective dissemination and training assistance to ensure that current and future generations of rehabilitation professionals are familiar with and skilled in the application of contemporary outcomes measurement.
1650-1977 (Print)Journal ArticleReview
Keywords: *Disability Evaluation; Activities of Daily Living/classification; Disabled Persons/classification/*rehabilitation; Health Status Indicators; Humans; Outcome Assessment (Health Care)/*methods/standards; Recovery of Function; Research Support, N.I.H., Extramural; Research Support, U.S. Gov't, Non-P.H.S.; Sensitivity and Specificity computerized adaptive testing

Controlling item exposure and test overlap in computerized adaptive testing - Chen, S.Y.; Lei, P. W. (Applied Psychological Measurement, 2005)

Controlling Item Exposure and Test Overlap in Computerized Adaptive Testing - Chen, Shu-Ying; Lei, Pui-Wa (Applied Psychological Measurement, 2005)
This article proposes an item exposure control method, which is the extension of the Sympson and Hetter procedure and can provide item exposure control at both the item and test levels. Item exposure rate and test overlap rate are two indices commonly used to track item exposure in computerized adaptive tests. By considering both indices, item exposure can be monitored at both the item and test levels. To control the item exposure rate and test overlap rate simultaneously, the modified procedure attempted to control not only the maximum value but also the variance of item exposure rates. Results indicated that the item exposure rate and test overlap rate could be controlled simultaneously by implementing the modified procedure. Item exposure control was improved and precision of trait estimation decreased when a prespecified maximum test overlap rate was stringent.

Controlling item exposure and test overlap in computerized adaptive testing - Chen, S-Y.; Lei, P-W. (Applied Psychological Measurement, 2005)
This article proposes an item exposure control method, which is the extension of the Sympson and Hetter procedure and can provide item exposure control at both the item and test levels. Item exposure rate and test overlap rate are two indices commonly used to track item exposure in computerized adaptive tests. By considering both indices, item exposure can be monitored at both the item and test levels. To control the item exposure rate and test overlap rate simultaneously, the modified procedure attempted to control not only the maximum value but also the variance of item exposure rates. Results indicated that the item exposure rate and test overlap rate could be controlled simultaneously by implementing the modified procedure. Item exposure control was improved and precision of trait estimation decreased when a prespecified maximum test overlap rate was stringent. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Adaptive Testing; Computer Assisted Testing; Item Content (Test) computerized adaptive testing

Data pooling and analysis to build a preliminary item bank: an example using bowel function in prostate cancer - Eton, D. T.; Lai, J. S.; Cella, D.; Reeve, B. B.; Talcott, J. A.; Clark, J. A.; McPherson, C. P.; Litwin, M. S.; Moinpour, C. M. (Evaluation and the Health Professions, 2005)
Assessing bowel function (BF) in prostate cancer can help determine therapeutic trade-offs. We determined the components of BF commonly assessed in prostate cancer studies as an initial step in creating an item bank for clinical and research application. We analyzed six archived data sets representing 4,246 men with prostate cancer. Thirty-one items from validated instruments were available for analysis. Items were classified into domains (diarrhea, rectal urgency, pain, bleeding, bother/distress, and other) then subjected to conventional psychometric and item response theory (IRT) analyses. Items fit the IRT model if the ratio between observed and expected item variance was between 0.60 and 1.40. Four of 31 items had inadequate fit in at least one analysis. Poorly fitting items included bleeding (2), rectal urgency (1), and bother/distress (1). A fifth item assessing hemorrhoids was poorly correlated with other items. Our analyses supported four related components of BF: diarrhea, rectal urgency, pain, and bother/distress.
0163-2787 (Print)Journal Article
Keywords: *Quality of Life; *Questionnaires; Adult; Aged; Data Collection/methods; Humans; Intestine, Large/*physiopathology; Male; Middle Aged; Prostatic Neoplasms/*physiopathology; Psychometrics; Research Support, Non-U.S. Gov't; Statistics, Nonparametric

Design and evaluation of an XML-based platform-independent computerized adaptive testing system - Ho, R.-G.,; Yen, Y.-C. (IEEE Transactions on Education, 2005)

Development of a computer-adaptive test for depression (D-CAT) - Fliege, H.; Becker, J.; Walter, O. B.; Bjorner, J. B.; Klapp, B. F.; Rose, M. (Quality of Life Research, 2005)

The development of the adaptive item language assessment (AILA) for mixed-ability students - Giouroglou, H.; Economides, A. A. (2005)

Dynamic assessment of health outcomes: Time to let the CAT out of the bag? - Cook, K. F.; O'Malley, K. J.; Roddey, T. S. (Health Services Research, 2005)
Background: The use of item response theory (IRT) to measure self-reported outcomes has burgeoned in recent years. Perhaps the most important application of IRT is computer-adaptive testing (CAT), a measurement approach in which the selection of items is tailored for each respondent. Objective. To provide an introduction to the use of CAT in the measurement of health outcomes, describe several IRT models that can be used as the basis of CAT, and discuss practical issues associated with the use of adaptive scaling in research settings. Principal Points: The development of a CAT requires several steps that are not required in the development of a traditional measure including identification of "starting" and "stopping" rules. CAT's most attractive advantage is its efficiency. Greater measurement precision can be achieved with fewer items. Disadvantages of CAT include the high cost and level of technical expertise required to develop a CAT. Conclusions: Researchers, clinicians, and patients benefit from the availability of psychometrically rigorous measures that are not burdensome. CAT outcome measures hold substantial promise in this regard, but their development is not without challenges. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: computer adaptive testing; Item Response Theory; self reported health outcomes

The effectiveness of using multiple item pools in computerized adaptive testing - Zhang, J.; Chang, H. (Annual meeting of the National Council on Measurement in Education, 2005)

Features of the estimated sampling distribution of the ability estimate in computerized adaptive testing according to two stopping rules - Blais, J-G.; Raîche, G. (2005)

Identifying practical indices for enhancing item pool security - Yi, Q.; Zhang, J.; Chang, Hua-Hua (Paper presented at the annual meeting of the National Council on Measurement in Education (NCME), 2005)

An implemented theoretical framework for a common European foreign language adaptive assessment - Giouroglou, H.; Economides, A. A. (2005)

Implementing content constraints in alpha-stratified adaptive testing using a shadow test approach - van der Linden, W. J.; Chang, Hua-Hua (2005)

Increasing the homogeneity of CAT's item-exposure rates by minimizing or maximizing varied target functions while assembling shadow tests - Li, Y. H.; Schafer, W. D. (Journal of Educational Measurement, 2005)
A computerized adaptive testing (CAT) algorithm that has the potential to increase the homogeneity of CATs item-exposure rates without significantly sacrificing the precision of ability estimates was proposed and assessed in the shadow-test (van der Linden & Reese, 1998) CAT context. This CAT algorithm was formed by a combination of maximizing or minimizing varied target functions while assembling shadow tests. There were four target functions to be separately used in the first, second, third, and fourth quarter test of CAT. The elements to be used in the four functions were associated with (a) a random number assigned to each item, (b) the absolute difference between an examinee's current ability estimate and an item difficulty, (c) the absolute difference between an examinee's current ability estimate and an optimum item difficulty, and (d) item information. The results indicated that this combined CAT fully utilized all the items in the pool, reduced the maximum exposure rates, and achieved more homogeneous exposure rates. Moreover, its precision in recovering ability estimates was similar to that of the maximum item-information method. The combined CAT method resulted in the best overall results compared with the other individual CAT item-selection methods. The findings from the combined CAT are encouraging. Future uses are discussed. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: algorithm; computerized adaptive testing; item exposure rate; shadow test; varied target function

Infeasibility in automated test assembly models: A comparison study of different methods - Huitzing, H. A.; Veldkamp, B. P.; Verschoor, A. J. (Journal of Educational Measurement, 2005)
Several techniques exist to automatically put together a test meeting a number of specifications. In an item bank, the items are stored with their characteristics. A test is constructed by selecting a set of items that fulfills the specifications set by the test assembler. Test assembly problems are often formulated in terms of a model consisting of restrictions and an objective to be maximized or minimized. A problem arises when it is impossible to construct a test from the item pool that meets all specifications, that is, when the model is not feasible. Several methods exist to handle these infeasibility problems. In this article, test assembly models resulting from two practical testing programs were reconstructed to be infeasible. These models were analyzed using methods that forced a solution (Goal Programming, Multiple-Goal Programming, Greedy Heuristic), that analyzed the causes (Relaxed and Ordered Deletion Algorithm (RODA), Integer Randomized Deletion Algorithm (IRDA), Set Covering (SC), and Item Sampling), or that analyzed the causes and used this information to force a solution (Irreducible Infeasible Set-Solver). Specialized methods such as the IRDA and the Irreducible Infeasible Set-Solver performed best. Recommendations about the use of different methods are given. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Algorithms; Item Content (Test); Models; Test Construction

An item bank was created to improve the measurement of cancer-related fatigue - Lai, J-S.; Cella, D.; Dineen, K.; Bode, R.; Von Roenn, J.; Gershon, R. C.; Shevrin, D. (Journal of Clinical Epidemiology, 2005)
OBJECTIVE: Cancer-related fatigue (CRF) is one of the most common unrelieved symptoms experienced by patients. CRF is underrecognized and undertreated due to a lack of clinically sensitive instruments that integrate easily into clinics. Modern computerized adaptive testing (CAT) can overcome these obstacles by enabling precise assessment of fatigue without requiring the administration of a large number of questions. A working item bank is essential for development of a CAT platform. The present report describes the building of an operational item bank for use in clinical settings with the ultimate goal of improving CRF identification and treatment. STUDY DESIGN AND SETTING: The sample included 301 cancer patients. Psychometric properties of items were examined by using Rasch analysis, an Item Response Theory (IRT) model. RESULTS AND CONCLUSION: The final bank includes 72 items. These 72 unidimensional items explained 57.5% of the variance, based on factor analysis results. Excellent internal consistency (alpha=0.99) and acceptable item-total correlation were found (range: 0.51-0.85). The 72 items covered a reasonable range of the fatigue continuum. No significant ceiling effects, floor effects, or gaps were found. A sample short form was created for demonstration purposes. The resulting bank is amenable to the development of a CAT platform.
Lai, Jin-SheiCella, DavidDineen, KellyBode, RitaVon Roenn, JamieGershon, Richard CShevrin, DanielEnglandJ Clin Epidemiol. 2005 Feb;58(2):190-7.
Keywords: Adult; Aged; Aged, 80 and over; Factor Analysis, Statistical; Fatigue/*etiology/psychology; Female; Humans; Male; Middle Aged; Neoplasms/*complications/psychology; Psychometrics; Questionnaires

[Item characteristic curve equating under graded response models in IRT] - Jun, Z.; Dongming, O.; Shuyuan, X.; Haiqi, D.; Shuqing, Q. (Acta Psychologica Sinica, 2005)
In one of the largest qualificatory tests--economist test, to guarantee the comparability among different years, construct item bank and prepare for computerized adaptive testing, item characteristic curve equating and anchor test equating design under graded models in IRT are used, which have realized the item and ability parameter equating of test data in five years and succeeded in establishing an item bank. Based on it, cut scores of different years are compared by equating and provide demonstrational gist to constitute the eligibility standard of economist test.
Keywords: graded response models; item characteristic curve; Item Response Theory

Item response theory in computer adaptive testing: implications for outcomes measurement in rehabilitation - Ware, J. E; Gandek, B.; Sinclair, S. J.; Bjorner, J. B. (Rehabil Psychol, 2005)

An item response theory-based pain item bank can enhance measurement precision - Lai, J-S.; Dineen, K.; Reeve, B. B.; Von Roenn, J.; Shervin, D.; McGuire, M.; Bode, R. K.; Paice, J.; Cella, D. (Journal of Pain and Symptom Management, 2005)
Cancer-related pain is often under-recognized and undertreated. This is partly due to the lack of appropriate assessments, which need to be comprehensive and precise yet easily integrated into clinics. Computerized adaptive testing (CAT) can enable precise-yet-brief assessments by only selecting the most informative items from a calibrated item bank. The purpose of this study was to create such a bank. The sample included 400 cancer patients who were asked to complete 61 pain-related items. Data were analyzed using factor analysis and the Rasch model. The final bank consisted of 43 items which satisfied the measurement requirement of factor analysis and the Rasch model, demonstrated high internal consistency and reasonable item-total correlations, and discriminated patients with differing degrees of pain. We conclude that this bank demonstrates good psychometric properties, is sensitive to pain reported by patients, and can be used as the foundation for a CAT pain-testing platform for use in clinical practice.
0885-3924Journal Article
Keywords: computerized adaptive testing

La Validez desde una óptica psicométrica [Validity from a psychometric perspective] - Muñiz, J. (Acta Comportamentalia, 2005)
El estudio de la validez constituye el eje central de los análisis psicométricos de los instrumentos de medida. En esta comunicación se traza una breve nota histórica de los distintos modos de concebir la validez a lo largo de los tiempos, se comentan las líneas actuales, y se tratan de vislumbrar posibles vías futuras, teniendo en cuenta el impacto que las nuevas tecnologías informáticas están ejerciendo sobre los propios instrumentos de medida en Psicología y Educación. Cuestiones como los nuevos formatos multimedia de los ítems, la evaluación a distancia, el uso intercultural de las pruebas, las consecuencias de su uso, o los tests adaptativos informatizados, reclaman nuevas formas de evaluar y conceptualizar la validez. También se analizan críticamente algunos planteamientos recientes sobre el concepto de validez. The study of validity constitutes a central axis of psychometric analyses of measurement instruments. This paper presents a historical sketch of different modes of conceiving validity, with commentary on current views, and it attempts to predict future lines of research by considering the impact of new computerized technologies on measurement instruments in psychology and education. Factors such as the new multimedia format of items, distance assessment, the intercultural use of tests, the consequences of the latter, or the development of computerized adaptive tests demand new ways of conceiving and evaluating validity. Some recent thoughts about the concept of validity are also critically analyzed. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Factor Analysis; Measurement; Psychometrics; Scaling (Testing); Statistical; Technology; Test Validity

Measuring physical function in patients with complex medical and postsurgical conditions: a computer adaptive approach - Siebens, H.; Andres, P. L.; Pengsheng, N.; Coster, W. J.; Haley, S. M. (American Journal of Physical Medicine and Rehabilitation, 2005)
OBJECTIVE: To examine whether the range of disability in the medically complex and postsurgical populations receiving rehabilitation is adequately sampled by the new Activity Measure--Post-Acute Care (AM-PAC), and to assess whether computer adaptive testing (CAT) can derive valid patient scores using fewer questions. DESIGN: Observational study of 158 subjects (mean age 67.2 yrs) receiving skilled rehabilitation services in inpatient (acute rehabilitation hospitals, skilled nursing facility units) and community (home health services, outpatient departments) settings for recent-onset or worsening disability from medical (excluding neurological) and surgical (excluding orthopedic) conditions. Measures were interviewer-administered activity questions (all patients) and physical functioning portion of the SF-36 (outpatients) and standardized chart items (11 Functional Independence Measure (FIM), 19 Standardized Outcome and Assessment Information Set (OASIS) items, and 22 Minimum Data Set (MDS) items). Rasch modeling analyzed all data and the relationship between person ability estimates and average item difficulty. CAT assessed the ability to derive accurate patient scores using a sample of questions. RESULTS: The 163-item activity item pool covered the range of physical movement and personal and instrumental activities. CAT analysis showed comparable scores between estimates using 10 items or the total item pool. CONCLUSION: The AM-PAC can assess a broad range of function in patients with complex medical illness. CAT achieves valid patient scores using fewer questions.
0894-9115 (Print)Comparative StudyJournal ArticleResearch Support, N.I.H., ExtramuralResearch Support, U.S. Gov't, P.H.S.
Keywords: Activities of Daily Living/*classification; Adult; Aged; Cohort Studies; Continuity of Patient Care; Disability Evaluation; Female; Health Services Research; Humans; Male; Middle Aged; Postoperative Care/*rehabilitation; Prognosis; Recovery of Function; Rehabilitation Centers; Rehabilitation/*standards; Sensitivity and Specificity; Sickness Impact Profile; Treatment Outcome

Monte Carlo Test Assembly for Item Pool Analysis and Extension - Belov, Dmitry I.; Armstrong, Ronald D. (Applied Psychological Measurement, 2005)
A new test assembly algorithm based on a Monte Carlo random search is presented in this article. A major advantage of the Monte Carlo test assembly over other approaches (integer programming or enumerative heuristics) is that it performs a uniform sampling from the item pool, which provides every feasible item combination (test) with an equal chance of being built during an assembly. This allows the authors to address the following issues of pool analysis and extension: compare the strengths and weaknesses of different pools, identify the most restrictive constraint(s) for test assembly, and identify properties of the items that should be added to a pool to achieve greater usability of the pool. Computer experiments with operational pools are given.

Personalized feedback in CAT - Economides, A. A. (2005)

The promise of PROMIS: using item response theory to improve assessment of patient-reported outcomes - Fries, J.F.; Bruce, B.; Cella, D. (Clinical and Experimental Rheumatology, 2005)
PROMIS (Patient-Reported-Outcomes Measurement Information System) is an NIH Roadmap network project intended to improve the reliability, validity, and precision of PROs and to provide definitive new instruments that will exceed the capabilities of classic instruments and enable improved outcome measurement for clinical research across all NIH institutes. Item response theory (IRT) measurement models now permit us to transition conventional health status assessment into an era of item banking and computerized adaptive testing (CAT). Item banking uses IRT measurement models and methods to develop item banks from large pools of items from many available questionnaires. IRT allows the reduction and improvement of items and assembles domains of items which are unidimensional and not excessively redundant. CAT provides a model-driven algorithm and software to iteratively select the most informative remaining item in a domain until a desired degree of precision is obtained. Through these approaches the number of patients required for a clinical trial may be reduced while holding statistical power constant. PROMIS tools, expected to improve precision and enable assessment at the individual patient level which should broaden the appeal of PROs, will begin to be available to the general medical community in 2008.
Keywords: computerized adaptive testing

Propiedades psicométricas de un test Adaptativo Informatizado para la medición del ajuste emocional [Psychometric properties of an Emotional Adjustment Computerized Adaptive Test] - Aguado, D.; Rubio, V. J.; Hontangas, P. M.; Hernández, J. M. (Psicothema, 2005)
En el presente trabajo se describen las propiedades psicométricas de un Test Adaptativo Informatizado para la medición del ajuste emocional de las personas. La revisión de la literatura acerca de la aplicación de los modelos de la teoría de la respuesta a los ítems (TRI) muestra que ésta se ha utilizado más en el trabajo con variables aptitudinales que para la medición de variables de personalidad, sin embargo diversos estudios han mostrado la eficacia de la TRI para la descripción psicométrica de dichasvariables. Aun así, pocos trabajos han explorado las características de un Test Adaptativo Informatizado, basado en la TRI, para la medición de una variable de personalidad como es el ajuste emocional. Nuestros resultados muestran la eficiencia del TAI para la evaluación del ajuste emocional, proporcionando una medición válida y precisa, utilizando menor número de elementos de medida encomparación con las escalas de ajuste emocional de instrumentos fuertemente implantados. Psychometric properties of an emotional adjustment computerized adaptive test. In the present work it was described the psychometric properties of an emotional adjustment computerized adaptive test. An examination of Item Response Theory (IRT) research literature indicates that IRT has been mainly used for assessing achievements and ability rather than personality factors. Nevertheless last years have shown several studies wich have successfully used IRT to personality assessment instruments. Even so, a few amount of works has inquired the computerized adaptative test features, based on IRT, for the measurement of a personality traits as it’s the emotional adjustment. Our results show the CAT efficiency for the emotional adjustment assessment so this provides a valid and accurate measurement; by using a less number of items in comparison with the emotional adjustment scales from the most strongly established questionnaires.
Keywords: Computer Assisted Testing; Emotional Adjustment; Item Response; Personality Measures; Psychometrics; Test Validity; Theory

A randomized experiment to compare conventional, computerized, and computerized adaptive administration of ordinal polytomous attitude items - Hol, A. M.; Vorst, H. C. M.; Mellenbergh, G. J. (Applied Psychological Measurement, 2005)
A total of 520 high school students were randomly assigned to a paper-and-pencil test (PPT), a computerized standard test (CST), or a computerized adaptive test (CAT) version of the Dutch School Attitude Questionnaire (SAQ), consisting of ordinal polytomous items. The CST administered items in the same order as the PPT. The CAT administered all items of three SAQ subscales in adaptive order using Samejima's graded response model, so that six different stopping rule settings could be applied afterwards. School marks were used as external criteria. Results showed significant but small multivariate administration mode effects on conventional raw scores and small to medium effects on maximum likelihood latent trait estimates. When the precision of CAT latent trait estimates decreased, correlations with grade point average in general decreased. However, the magnitude of the decrease was not very large as compared to the PPT, the CST, and the CAT without the stopping rule. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Computer Assisted Testing; Test Administration; Test Items

A Randomized Experiment to Compare Conventional, Computerized, and Computerized Adaptive Administration of Ordinal Polytomous Attitude Items - Hol, A. Michiel; Vorst, Harrie C. M.; Mellenbergh, Gideon J. (Applied Psychological Measurement, 2005)
A total of 520 high school students were randomly assigned to a paper-and-pencil test (PPT), a computerized standard test (CST), or a computerized adaptive test (CAT) version of the Dutch School Attitude Questionnaire (SAQ), consisting of ordinal polytomous items. The CST administered items in the same order as the PPT. The CAT administered all items of three SAQ subscales in adaptive order using Samejima’s graded response model, so that six different stopping rule settings could be applied afterwards. School marks were used as external criteria. Results showed significant but small multivariate administration mode effects on conventional raw scores and small to medium effects on maximum likelihood latent trait estimates. When the precision of CAT latent trait estimates decreased, correlations with grade point average in general decreased. However, the magnitude of the decrease was not very large as compared to the PPT, the CST, and the CAT without the stopping rule.

Recent trends in comparability studies - Paek, P. (PEM Research Report 05-05, 2005)
Keywords: computer adaptive testing; Computerized assessment; differential item functioning; Mode effects

Rescuing CAT by fixing the problems - Chang, S-H.; Zhang, J. (National Council on Measurement in Education, 2005)

Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments - Hart, D. L.; Mioduski, J. E.; Stratford, P. W. (Journal of Clinical Epidemiology, 2005)
BACKGROUND AND OBJECTIVE: To develop computerized adaptive tests (CATs) designed to assess lower extremity functional status (FS) in people with lower extremity impairments using items from the Lower Extremity Functional Scale and compare discriminant validity of FS measures generated using all items analyzed with a rating scale Item Response Theory model (theta(IRT)) and measures generated using the simulated CATs (theta(CAT)). METHODS: Secondary analysis of retrospective intake rehabilitation data. RESULTS: Unidimensionality of items was strong, and local independence of items was adequate. Differential item functioning (DIF) affected item calibration related to body part, that is, hip, knee, or foot/ankle, but DIF did not affect item calibration for symptom acuity, gender, age, or surgical history. Therefore, patients were separated into three body part specific groups. The rating scale model fit all three data sets well. Three body part specific CATs were developed: each was 70% more efficient than using all LEFS items to estimate FS measures. theta(IRT) and theta(CAT) measures discriminated patients by symptom acuity, age, and surgical history in similar ways. theta(CAT) measures were as precise as theta(IRT) measures. CONCLUSION: Body part-specific simulated CATs were efficient and produced precise measures of FS with good discriminant validity.
0895-4356 (Print)Journal ArticleMulticenter StudyValidation Studies
Keywords: *Health Status Indicators; Activities of Daily Living; Adolescent; Adult; Aged; Aged, 80 and over; Ankle Joint/physiopathology; Diagnosis, Computer-Assisted/*methods; Female; Hip Joint/physiopathology; Humans; Joint Diseases/physiopathology/*rehabilitation; Knee Joint/physiopathology; Lower Extremity/*physiopathology; Male; Middle Aged; Research Support, N.I.H., Extramural; Research Support, U.S. Gov't, P.H.S.; Retrospective Studies

Somministrazione di test computerizzati di tipo adattivo: Un' applicazione del modello di misurazione di Rasch [Administration of computerized and adaptive tests: An application of the Rasch Model] - Miceli, R.; Molinengo, G. (Testing Psicometria Metodologia, 2005)
The aim of the present study is to describe the characteristics of a procedure for administering computerized and adaptive tests (Computer Adaptive Testing or CAT). Items to be asked to the individuals are interactively chosen and are selected from a "bank" in which they were previously calibrated and recorded on the basis of their difficulty level. The selection of items is performed by increasingly more accurate estimates of the examinees' ability. The building of an item-bank on Psychometrics and the implementation of this procedure allow a first validation through Monte Carlo simulations. (PsycINFO Database Record (c) 2006 APA ) (journal abstract)
Keywords: Adaptive Testing; Computer Assisted Testing; Item Response Theory computerized adaptive testing; Models; Psychometrics

Strategies for controlling item exposure in computerized adaptive testing with the partial credit model - Davis, L. L.; Dodd, B. (2005)

Test construction for cognitive diagnosis - Henson, R. K.; Douglas, J. (Applied Psychological Measurement, 2005)
Although cognitive diagnostic models (CDMs) can be useful in the analysis and interpretation of existing tests, little has been developed to specify how one might construct a good test using aspects of the CDMs. This article discusses the derivation of a general CDM index based on Kullback-Leibler information that will serve as a measure of how informative an item is for the classification of examinees. The effectiveness of the index is examined for items calibrated using the deterministic input noisy "and" gate model (DINA) and the reparameterized unified model (RUM) by implementing a simple heuristic to construct a test from an item bank. When compared to randomly constructed tests from the same item bank, the heuristic shows significant improvement in classification rates. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: (Measurement); Cognitive Assessment; Item Analysis (Statistical); Profiles; Test Construction; Test Interpretation; Test Items

Toward efficient and comprehensive measurement of the alcohol problems continuum in college students: The Brief Young Adult Alcohol Consequences Questionnaire - Kahler, C. W.; Strong, D. R.; Read, J. P.; De Boeck, P.; Wilson, M.; Acton, G. S.; Palfai, T. P.; Wood, M. D.; Mehta, P. D.; Neale, M. C.; Flay, B. R.; Conklin, C. A.; et al. (Alcoholism: Clinical & Experimental Research, 2005)
Background: Although a number of measures of alcohol problems in college students have been studied, the psychometric development and validation of these scales have been limited, for the most part, to methods based on classical test theory. In this study, we conducted analyses based on item response theory to select a set of items for measuring the alcohol problem severity continuum in college students that balances comprehensiveness and efficiency and is free from significant gender bias., Method: We conducted Rasch model analyses of responses to the 48-item Young Adult Alcohol Consequences Questionnaire by 164 male and 176 female college students who drank on at least a weekly basis. An iterative process using item fit statistics, item severities, item discrimination parameters, model residuals, and analysis of differential item functioning by gender was used to pare the items down to those that best fit a Rasch model and that were most efficient in discriminating among levels of alcohol problems in the sample., Results: The process of iterative Rasch model analyses resulted in a final 24-item scale with the data fitting the unidimensional Rasch model very well. The scale showed excellent distributional properties, had items adequately matched to the severity of alcohol problems in the sample, covered a full range of problem severity, and appeared highly efficient in retaining all of the meaningful variance captured by the original set of 48 items., Conclusions: The use of Rasch model analyses to inform item selection produced a final scale that, in both its comprehensiveness and its efficiency, should be a useful tool for researchers studying alcohol problems in college students. To aid interpretation of raw scores, examples of the types of alcohol problems that are likely to be experienced across a range of selected scores are provided., (C)2005Research Society on AlcoholismAn important, sometimes controversial feature of all psychological phenomena is whether they are categorical or dimensional. A conceptual and psychometric framework is described for distinguishing whether the latent structure behind manifest categories (e.g., psychiatric diagnoses, attitude groups, or stages of development) is category-like or dimension-like. Being dimension-like requires (a) within-category heterogeneity and (b) between-category quantitative differences. Being category-like requires (a) within-category homogeneity and (b) between-category qualitative differences. The relation between this classification and abrupt versus smooth differences is discussed. Hybrid structures are possible. Being category-like is itself a matter of degree; the authors offer a formalized framework to determine this degree. Empirical applications to personality disorders, attitudes toward capital punishment, and stages of cognitive development illustrate the approach., (C) 2005 by the American Psychological AssociationThe authors conducted Rasch model ( G. Rasch, 1960) analyses of items from the Young Adult Alcohol Problems Screening Test (YAAPST; S. C. Hurlbut & K. J. Sher, 1992) to examine the relative severity and ordering of alcohol problems in 806 college students. Items appeared to measure a single dimension of alcohol problem severity, covering a broad range of the latent continuum. Items fit the Rasch model well, with less severe symptoms reliably preceding more severe symptoms in a potential progression toward increasing levels of problem severity. However, certain items did not index problem severity consistently across demographic subgroups. A shortened, alternative version of the YAAPST is proposed, and a norm table is provided that allows for a linking of total YAAPST scores to expected symptom expression., (C) 2004 by the American Psychological AssociationA didactic on latent growth curve modeling for ordinal outcomes is presented. The conceptual aspects of modeling growth with ordinal variables and the notion of threshold invariance are illustrated graphically using a hypothetical example. The ordinal growth model is described in terms of 3 nested models: (a) multivariate normality of the underlying continuous latent variables (yt) and its relationship with the observed ordinal response pattern (Yt), (b) threshold invariance over time, and (c) growth model for the continuous latent variable on a common scale. Algebraic implications of the model restrictions are derived, and practical aspects of fitting ordinal growth models are discussed with the help of an empirical example and Mx script ( M. C. Neale, S. M. Boker, G. Xie, & H. H. Maes, 1999). The necessary conditions for the identification of growth models with ordinal data and the methodological implications of the model of threshold invariance are discussed., (C) 2004 by the American Psychological AssociationRecent research points toward the viability of conceptualizing alcohol problems as arrayed along a continuum. Nevertheless, modern statistical techniques designed to scale multiple problems along a continuum (latent trait modeling; LTM) have rarely been applied to alcohol problems. This study applies LTM methods to data on 110 problems reported during in-person interviews of 1,348 middle-aged men (mean age = 43) from the general population. The results revealed a continuum of severity linking the 110 problems, ranging from heavy and abusive drinking, through tolerance and withdrawal, to serious complications of alcoholism. These results indicate that alcohol problems can be arrayed along a dimension of severity and emphasize the relevance of LTM to informing the conceptualization and assessment of alcohol problems., (C) 2004 by the American Psychological AssociationItem response theory (IRT) is supplanting classical test theory as the basis for measures development. This study demonstrated the utility of IRT for evaluating DSM-IV diagnostic criteria. Data on alcohol, cannabis, and cocaine symptoms from 372 adult clinical participants interviewed with the Composite International Diagnostic Interview-Expanded Substance Abuse Module (CIDI-SAM) were analyzed with Mplus ( B. Muthen & L. Muthen, 1998) and MULTILOG ( D. Thissen, 1991) software. Tolerance and legal problems criteria were dropped because of poor fit with a unidimensional model. Item response curves, test information curves, and testing of variously constrained models suggested that DSM-IV criteria in the CIDI-SAM discriminate between only impaired and less impaired cases and may not be useful to scale case severity. IRT can be used to study the construct validity of DSM-IV diagnoses and to identify diagnostic criteria with poor performance., (C) 2004 by the American Psychological AssociationThis study examined the psychometric characteristics of an index of substance use involvement using item response theory. The sample consisted of 292 men and 140 women who qualified for a Diagnostic and Statistical Manual of Mental Disorders (3rd ed., rev.; American Psychiatric Association, 1987) substance use disorder (SUD) diagnosis and 293 men and 445 women who did not qualify for a SUD diagnosis. The results indicated that men had a higher probability of endorsing substance use compared with women. The index significantly predicted health, psychiatric, and psychosocial disturbances as well as level of substance use behavior and severity of SUD after a 2-year follow-up. Finally, this index is a reliable and useful prognostic indicator of the risk for SUD and the medical and psychosocial sequelae of drug consumption., (C) 2002 by the American Psychological AssociationComparability, validity, and impact of loss of information of a computerized adaptive administration of the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) were assessed in a sample of 140 Veterans Affairs hospital patients. The countdown method ( Butcher, Keller, & Bacon, 1985) was used to adaptively administer Scales L (Lie) and F (Frequency), the 10 clinical scales, and the 15 content scales. Participants completed the MMPI-2 twice, in 1 of 2 conditions: computerized conventional test-retest, or computerized conventional-computerized adaptive. Mean profiles and test-retest correlations across modalities were comparable. Correlations between MMPI-2 scales and criterion measures supported the validity of the countdown method, although some attenuation of validity was suggested for certain health-related items. Loss of information incurred with this mode of adaptive testing has minimal impact on test validity. Item and time savings were substantial., (C) 1999 by the American Psychological Association
MiscellaneousArticleMiscellaneous Article
Keywords: Psychometrics; Substance-Related Disorders

Trait Parameter Recovery Using Multidimensional Computerized Adaptive Testing in Reading and Mathematics - Li, Yuan H.; Schafer, William D. (Applied Psychological Measurement, 2005)
Under a multidimensional item response theory (MIRT) computerized adaptive testing (CAT) testing scenario, a trait estimate (θ) in one dimension will provide clues for subsequently seeking a solution in other dimensions. This feature may enhance the efficiency of MIRT CAT’s item selection and its scoring algorithms compared with its counterpart, the unidimensional CAT (UCAT). The present study used existing Reading and Math test data to generate simulated item parameters. A confirmatory item factor analysis model was applied to the data using NOHARM to produce interpretable MIRT item parameters. Results showed that MIRT CAT, conditional on the constraints, was quite capable of producing accurate estimates on both measures. Compared with UCAT, MIRT CAT slightly increased the accuracy of both trait estimates, especially for the low-level or high-level trait examinees in both measures, and reduced the rate of unused items in the item pool.

Trait parameter recovery using multidimensional computerized adaptive testing in reading and mathematics - Li, Y. H. (Applied Psychological Measurement, 2005)
Under a multidimensional item response theory (MIRT) computerized adaptive testing (CAT) testing scenario, a trait estimate (θ) in onedimension will provide clues for subsequentlyseeking a solution in other dimensions. Thisfeature may enhance the efficiency of MIRT CAT’s item selection and its scoring algorithms compared with its counterpart, the unidimensional CAT (UCAT). The present study used existing Reading and Math test data to generate simulated item parameters. A confirmatory item factor analysis model was applied to the data using NOHARM to produce interpretable MIRT item parameters. Results showed that MIRT CAT, conditional on theconstraints, was quite capable of producing accurate estimates on both measures. Compared with UCAT, MIRT CAT slightly increased the accuracy of both trait estimates, especially for the low-level or high-level trait examinees in both measures, and reduced the rate of unused items in the item pool. Index terms: computerized adaptive testing (CAT), item response theory (IRT), dimensionality, 0-1 linear programming, constraints, item exposure, reading assessment, mathematics assessment.

The use of person-fit statistics in computerized adaptive testing - Meijer, R. R.; van Krimpen-Stoop, E. M. L. A. (LSAC Research Report Series, 2005)

Validation of a computerized adaptive testing version of the Schedule for Nonadaptive and Adaptive Personality (SNAP) - Simms, L. J.,; Clark, L. A. (Psychological Assessment, 2005)
This is a validation study of a computerized adaptive (CAT) version of the Schedule for Nonadaptive and Adaptive Personality (SNAP) conducted with 413 undergraduates who completed the SNAP twice, 1 week apart. Participants were assigned randomly to 1 of 4 retest groups: (a) paper-and-pencil (P&P) SNAP, (b) CAT, (c) P&P/CAT, and (d) CAT/P&P. With number of items held constant, computerized administration had little effect on descriptive statistics, rank ordering of scores, reliability, and concurrent validity, but was preferred over P&P administration by most participants. CAT administration yielded somewhat lower precision and validity than P&P administration, but required 36% to 37% fewer items and 58% to 60% less time to complete. These results confirm not only key findings from previous CAT simulation studies of personality measures but extend them for the 1st time to a live assessment setting.

Validation of a computerized adaptive version of the Schedule of Non-Adaptive and Adaptive Personality (SNAP) - Simms, L. J.; Clark, L.J. (Psychological Assessment, 2005)
This is a validation study of a computerized adaptive (CAT) version of the Schedule for Nonadaptive and Adaptive Personality (SNAP) conducted with 413 undergraduates who completed the SNAP twice, 1 week apart. Participants were assigned randomly to 1 of 4 retest groups: (a) paper-and-pencil (P&P) SNAP, (b) CAT, (c) P&P/CAT, and (d) CAT/P&P. With number of items held constant, computerized administration had little effect on descriptive statistics, rank ordering of scores, reliability, and concurrent validity, but was preferred over P&P administration by most participants. CAT administration yielded somewhat lower precision and validity than P&P administration, but required 36% to 37% fewer items and 58% to 60% less time to complete. These results confirm not only key findings from previous CAT simulation studies of personality measures but extend them for the 1st time to a live assessment setting.

The ABCs of Computerized Adaptive Testing - Gershon, R. C. (2004)

Achieving accuracy of retest calibration for a national CAT placement examination with a restricted test length - Wang, X. B.; Wiley, A. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)
#WA04-01 {PDF file, 837 KB}

Activity outcome measurement for postacute care - Haley, S. M.; Coster, W. J.; Andres, P. L.; Ludlow, L. H.; Ni, P.; Bond, T. L.; Sinclair, S. J.; Jette, A. M. (Medical Care, 2004)
BACKGROUND: Efforts to evaluate the effectiveness of a broad range of postacute care services have been hindered by the lack of conceptually sound and comprehensive measures of outcomes. It is critical to determine a common underlying structure before employing current methods of item equating across outcome instruments for future item banking and computer-adaptive testing applications. OBJECTIVE: To investigate the factor structure, reliability, and scale properties of items underlying the Activity domains of the International Classification of Functioning, Disability and Health (ICF) for use in postacute care outcome measurement. METHODS: We developed a 41-item Activity Measure for Postacute Care (AM-PAC) that assessed an individual's execution of discrete daily tasks in his or her own environment across major content domains as defined by the ICF. We evaluated the reliability and discriminant validity of the prototype AM-PAC in 477 individuals in active rehabilitation programs across 4 rehabilitation settings using factor analyses, tests of item scaling, internal consistency reliability analyses, Rasch item response theory modeling, residual component analysis, and modified parallel analysis. RESULTS: Results from an initial exploratory factor analysis produced 3 distinct, interpretable factors that accounted for 72% of the variance: Applied Cognition (44%), Personal Care & Instrumental Activities (19%), and Physical & Movement Activities (9%); these 3 activity factors were verified by a confirmatory factor analysis. Scaling assumptions were met for each factor in the total sample and across diagnostic groups. Internal consistency reliability was high for the total sample (Cronbach alpha = 0.92 to 0.94), and for specific diagnostic groups (Cronbach alpha = 0.90 to 0.95). Rasch scaling, residual factor, differential item functioning, and modified parallel analyses supported the unidimensionality and goodness of fit of each unique activity domain. CONCLUSIONS: This 3-factor model of the AM-PAC can form the conceptual basis for common-item equating and computer-adaptive applications, leading to a comprehensive system of outcome instruments for postacute care settings.
0025-7079Journal ArticleMulticenter Study
Keywords: *Self Efficacy; *Sickness Impact Profile; Activities of Daily Living/*classification/psychology; Adult; Aftercare/*standards/statistics & numerical data; Aged; Boston; Cognition/physiology; Disability Evaluation; Factor Analysis, Statistical; Female; Human; Male; Middle Aged; Movement/physiology; Outcome Assessment (Health Care)/*methods/statistics & numerical data; Psychometrics; Questionnaires/standards; Rehabilitation/*standards/statistics & numerical data; Reproducibility of Results; Sensitivity and Specificity; Support, U.S. Gov't, Non-P.H.S.; Support, U.S. Gov't, P.H.S.

Adaptive computerized educational systems: A case study - Ray, R. D. (Evidence-based educational methods, 2004)
(Created by APA) Adaptive instruction describes adjustments typical of one-on-one tutoring as discussed in the college tutorial scenario. So computerized adaptive instruction refers to the use of computer software--almost always incorporating artificially intelligent services--which has been designed to adjust both the presentation of information and the form of questioning to meet the current needs of an individual learner. This chapter describes a system for Internet-delivered adaptive instruction. The author attempts to demonstrate a sharp difference between the teaching that takes place outside of the classroom in universities and the kind that is at least afforded, if not taken advantage of by many, students in a more personalized educational setting such as those in the small liberal arts colleges. The author describes a computer-based technology that allows that gap to be bridged with the advantage of at least having more highly prepared learners sitting in college classrooms. A limited range of emerging research that supports that proposition is cited. (PsycINFO Database Record (c) 2005 APA )
Using Smart Source ParsingEvidence-based educational methods. A volume in the educational psychology series. (pp. 143-170). San Diego, CA : Elsevier Academic Press, [URL:http://www.academicpress.com]. xxiv, 382 pp
Keywords: Artificial; Computer Assisted Instruction; Computer Software; Higher Education; Individualized; Instruction; Intelligence; Internet; Undergraduate Education

Adaptive exploration of user knowledge in computer based testing - Lamboudis, D.; Economides, A. A. (WSEAS Transactions on Communications, 2004)

Adaptive Testing With Regression Trees in the Presence of Multidimensionality - Yan, Duanli; Lewis, Charles; Stocking, Martha (Journal of Educational and Behavioral Statistics, 2004)
It is unrealistic to suppose that standard item response theory (IRT) models will be appropriate for all the new and currently considered computer-based tests. In addition to developing new models, we also need to give attention to the possibility of constructing and analyzing new tests without the aid of strong models. Computerized adaptive testing currently relies heavily on IRT. Alternative, empirically based, nonparametric adaptive testing algorithms exist, but their properties are little known. This article introduces a nonparametric, tree-based algorithm for adaptive testing and shows that it may be superior to conventional, IRT-based adaptive testing in cases where the IRT assumptions are not satisfied. In particular, it shows that the tree-based approach clearly outperformed (one-dimensional) IRT when the pool was strongly two-dimensional.

The AMC Linear Disability Score project in a population requiring residential care: psychometric properties - Holman, R.; Lindeboom, R.; Vermeulen, M.; de Haan, R. J. (Health and Quality of Life Outcomes, 2004)
BACKGROUND: Currently there is a lot of interest in the flexible framework offered by item banks for measuring patient relevant outcomes, including functional status. However, there are few item banks, which have been developed to quantify functional status, as expressed by the ability to perform activities of daily life. METHOD: This paper examines the psychometric properties of the AMC Linear Disability Score (ALDS) project item bank using an item response theory model and full information factor analysis. Data were collected from 555 respondents on a total of 160 items. RESULTS: Following the analysis, 79 items remained in the item bank. The remaining 81 items were excluded because of: difficulties in presentation (1 item); low levels of variation in response pattern (28 items); significant differences in measurement characteristics for males and females or for respondents under or over 85 years old (26 items); or lack of model fit to the data at item level (26 items). CONCLUSIONS: It is conceivable that the item bank will have different measurement characteristics for other patient or demographic populations. However, these results indicate that the ALDS item bank has sound psychometric properties for respondents in residential care settings and could form a stable base for measuring functional status in a range of situations, including the implementation of computerised adaptive testing of functional status.
Holman, RebeccaLindeboom, RobertVermeulen, Marinusde Haan, Rob JResearch Support, Non-U.S. Gov'tValidation StudiesEnglandHealth and quality of life outcomesHealth Qual Life Outcomes. 2004 Aug 3;2:42.
Keywords: *Disability Evaluation; *Health Status Indicators; Activities of Daily Living/*classification; Adult; Aged; Aged, 80 and over; Data Collection/methods; Female; Humans; Logistic Models; Male; Middle Aged; Netherlands; Pilot Projects; Probability; Psychometrics/*instrumentation; Questionnaires/standards; Residential Facilities/*utilization; Severity of Illness Index

The application of cognitive diagnosis and computerized adaptive testing to a large-scale assessment - McGlohen, MK (2004)

Assisted self-adapted testing: A comparative study - Hontangas, P.; Olea, J.; Ponsoda, V.; Revuelta, J.; Wise, S. L. (European Journal of Psychological Assessment, 2004)
A new type of self-adapted test (S-AT), called Assisted Self-Adapted Test (AS-AT), is presented. It differs from an ordinary S-AT in that prior to selecting the difficulty category, the computer advises examinees on their best difficulty category choice, based on their previous performance. Three tests (computerized adaptive test, AS-AT, and S-AT) were compared regarding both their psychometric (precision and efficiency) and psychological (anxiety) characteristics. Tests were applied in an actual assessment situation, in which test scores determined 20% of term grades. A sample of 173 high school students participated. Neither differences in posttest anxiety nor ability were obtained. Concerning precision, AS-AT was as precise as CAT, and both revealed more precision than S-AT. It was concluded that AS-AT acted as a CAT concerning precision. Some hints, but not conclusive support, of the psychological similarity between AS-AT and S-AT was also found. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Adaptive Testing; Anxiety; Computer Assisted Testing; Psychometrics; Test

Automated Simultaneous Assembly of Multi-Stage Testing for the Uniform CPA Examination - Breithaupt, K; Ariel, A.; Veldkamp, B. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)
{PDF file, 201 KB}

Combining computer adaptive testing technology with cognitively diagnostic assessment - McGlohen, MK; Chang, Hua-Hua; Wills, J. T. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)
{PDF file, 782 KB}

Computer adaptive testing: a strategy for monitoring stroke rehabilitation across settings - Andres, P. L.; Black-Schaffer, R. M.; Ni, P.; Haley, S. M. (Stroke Rehabilitation, 2004)
Current functional assessment instruments in stroke rehabilitation are often setting-specific and lack precision, breadth, and/or feasibility. Computer adaptive testing (CAT) offers a promising potential solution by providing a quick, yet precise, measure of function that can be used across a broad range of patient abilities and in multiple settings. CAT technology yields a precise score by selecting very few relevant items from a large and diverse item pool based on each individual's responses. We demonstrate the potential usefulness of a CAT assessment model with a cross-sectional sample of persons with stroke from multiple rehabilitation settings.
Andres, Patricia LBlack-Schaffer, Randie MNi, PengshengHaley, Stephen MR01 hd43568/hd/nichdEvaluation StudiesResearch Support, U.S. Gov't, Non-P.H.S.Research Support, U.S. Gov't, P.H.S.United StatesTopics in stroke rehabilitationTop Stroke Rehabil. 2004 Spring;11(2):33-9.
Keywords: *Computer Simulation; *User-Computer Interface; Adult; Aged; Aged, 80 and over; Cerebrovascular Accident/*rehabilitation; Disabled Persons/*classification; Female; Humans; Male; Middle Aged; Monitoring, Physiologic/methods; Severity of Illness Index; Task Performance and Analysis

Computer adaptive testing and the No Child Left Behind Act - Kingsbury, G. G.; Hauser, C. (Paper presented at the annual meeting of the American Educational Research Association, 2004)
{PDF file, 117 KB}

Computer-adaptive testing - Luecht, RM (2004)

Computer-based test designs with optimal and non-optimal tests for making pass-fail decisions - Hambleton, R. K.; Xing, D. (2004)

A computerized adaptive knowledge test as an assessment tool in general practice: a pilot study - Roex, A.; Degryse, J. (Medical Teacher, 2004)
Advantageous to assessment in many fields, CAT (computerized adaptive testing) use in general practice has been scarce. In adapting CAT to general practice, the basic assumptions of item response theory and the case specificity must be taken into account. In this context, this study first evaluated the feasibility of converting written extended matching tests into CAT. Second, it questioned the content validity of CAT. A stratified sample of students was invited to participate in the pilot study. The items used in this test, together with their parameters, originated from the written test. The detailed test paths of the students were retained and analysed thoroughly. Using the predefined pass-fail standard, one student failed the test. There was a positive correlation between the number of items and the candidate's ability level. The majority of students were presented with questions in seven of the 10 existing domains. Although proved to be a feasible test format, CAT cannot substitute for the existing high-stakes large-scale written test. It may provide a reliable instrument for identifying candidates who are at risk of failing in the written test.
0142-159xJournal Article
Keywords: *Computer Systems; Algorithms; Educational Measurement/*methods; Family Practice/*education; Humans; Pilot Projects

Computerized adaptive measurement of depression: A simulation study - Gardner, W.; Shear, K.; Kelleher, K. J.; Pajer, K. A.; Mammen, O.; Buysse, D.; Frank, E. (BMC Psychiatry, 2004)
Background: Efficient, accurate instruments for measuring depression are increasingly importantin clinical practice. We developed a computerized adaptive version of the Beck DepressionInventory (BDI). We examined its efficiency and its usefulness in identifying Major DepressiveEpisodes (MDE) and in measuring depression severity.Methods: Subjects were 744 participants in research studies in which each subject completed boththe BDI and the SCID. In addition, 285 patients completed the Hamilton Depression Rating Scale.Results: The adaptive BDI had an AUC as an indicator of a SCID diagnosis of MDE of 88%,equivalent to the full BDI. The adaptive BDI asked fewer questions than the full BDI (5.6 versus 21items). The adaptive latent depression score correlated r = .92 with the BDI total score and thelatent depression score correlated more highly with the Hamilton (r = .74) than the BDI total scoredid (r = .70).Conclusions: Adaptive testing for depression may provide greatly increased efficiency withoutloss of accuracy in identifying MDE or in measuring depression severity.
Keywords: *Computer Simulation; Adult; Algorithms; Area Under Curve; Comparative Study; Depressive Disorder/*diagnosis/epidemiology/psychology; Diagnosis, Computer-Assisted/*methods/statistics & numerical data; Factor Analysis, Statistical; Female; Humans; Internet; Male; Mass Screening/methods; Patient Selection; Personality Inventory/*statistics & numerical data; Pilot Projects; Prevalence; Psychiatric Status Rating Scales/*statistics & numerical data; Psychometrics; Research Support, Non-U.S. Gov't; Research Support, U.S. Gov't, P.H.S.; Severity of Illness Index; Software

Computerized adaptive testing - Segall, D. O. (2004)
{PDF file, 180 KB}

Computerized adaptive testing and item banking - Bjorner, J. B.; Kosinski, M.; Ware, J. E; Jr. (2004)
PDF file 371 KB

Computerized Adaptive Testing for Effective and Efficient Measurement in Counseling and Education - Weiss, D. J. (Measurement and Evaluation in Counseling and Development, 2004)

Computerized Adaptive Testing With Multiple-Form Structures - Armstrong, Ronald D.; Jones, Douglas H.; Koppel, Nicole B.; Pashley, Peter J. (Applied Psychological Measurement, 2004)
A multiple-form structure (MFS) is an orderedcollection or network of testlets (i.e., sets of items).An examinee’s progression through the networkof testlets is dictated by the correctness of anexaminee’s answers, thereby adapting the test tohis or her trait level. The collection of pathsthrough the network yields the set of all possibletest forms, allowing test specialists the opportunityto review them before they are administered. Also,limiting the exposure of an individual MFS to aspecific period of time can enhance test security.This article provides an overview of methods thathave been developed to generate parallel MFSs.The approach is applied to the assembly of anexperimental computerized Law School Admission Test (LSAT).

Computerized adaptive testing with multiple-form structures - Armstrong, R. D.; Jones, D. H.; Koppel, N. B.; Pashley, P. J. (Applied Psychological Measurement, 2004)
A multiple-form structure (MFS) is an ordered collection or network of testlets (i.e., sets of items). An examinee's progression through the network of testlets is dictated by the correctness of an examinee's answers, thereby adapting the test to his or her trait level. The collection of paths through the network yields the set of all possible test forms, allowing test specialists the opportunity to review them before they are administered. Also, limiting the exposure of an individual MFS to a specific period of time can enhance test security. This article provides an overview of methods that have been developed to generate parallel MFSs. The approach is applied to the assembly of an experimental computerized Law School Admission Test (LSAT). (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: computerized adaptive testing; Law School Admission Test; multiple-form structure; testlets

Computers in clinical assessment: Historical developments, present status, and future challenges - Butcher, J. N.; Perry, J. L.; Hahn, J. A. (Journal of Clinical Psychology, 2004)
Computerized testing methods have long been regarded as a potentially powerful asset for providing psychological assessment services. Ever since computers were first introduced and adapted to the field of assessment psychology in the 1950s, they have been a valuable aid for scoring, data processing, and even interpretation of test results. The history and status of computer-based personality and neuropsychological tests are discussed in this article. Several pertinent issues involved in providing test interpretation by computer are highlighted. Advances in computer-based test use, such as computerized adaptive testing, are described and problems noted. Today, there is great interest in expanding the availability of psychological assessment applications on the Internet. Although these applications show great promise, there are a number of problems associated with providing psychological tests on the Internet that need to be addressed by psychologists before the Internet can become a major medium for psychological service delivery. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: clinical assessment; computerized testing method; Internet; psychological assessment services

Constraining Item Exposure in Computerized Adaptive Testing With Shadow Tests - van der Linden, Wim J.; Veldkamp, Bernard P. (Journal of Educational and Behavioral Statistics, 2004)
Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter’s (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. The method does not, however, require time-consuming simulation studies to set values for control parameters before the operational use of the test. Instead, it can set the probabilities of item ineligibility adaptively during the test using the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test showed that application of the method yielded perfect control of the item-exposure rates and had negligible impact on the bias and mean-squared error functions of the ability estimator.

Constraining item exposure in computerized adaptive testing with shadow tests - van der Linden, W. J.; Veldkamp, B. P. (Journal of Educational and Behavioral Statistics, 2004)
Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter’s (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. The method does not, however, require time-consuming simulation studies to set values for control parameters before the operational use of the test. Instead, it can set the probabilities of item ineligibility adaptively during the test using the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test showed that application of the method yielded perfect control of the item-exposure rates and had negligible impact on the bias and mean-squared error functions of the ability estimator.
Keywords: computerized adaptive testing; item exposure control; item ineligibility constraints; Probability; shadow tests

Constructing rotating item pools for constrained adaptive testing - Ariel, A.; Veldkamp, B. P.; van der Linden, W. J. (Journal of Educational Measurement, 2004)
Preventing items in adaptive testing from being over- or underexposed is one of the main problems in computerized adaptive testing. Though the problem of overexposed items can be solved using a probabilistic item-exposure control method, such methods are unable to deal with the problem of underexposed items. Using a system of rotating item pools, on the other hand, is a method that potentially solves both problems. In this method, a master pool is divided into (possibly overlapping) smaller item pools, which are required to have similar distributions of content and statistical attributes. These pools are rotated among the testing sites to realize desirable exposure rates for the items. A test assembly model, motivated by Gulliksen's matched random subtests method, was explored to help solve the problem of dividing a master pool into a set of smaller pools. Different methods to solve the model are proposed. An item pool from the Law School Admission Test was used to evaluate the performances of computerized adaptive tests from systems of rotating item pools constructed using these methods. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: computerized adaptive tests; constrained adaptive testing; item exposure; rotating item pools

The context effects of multidimensional CAT on the accuracy of multidimensional abilities and the item exposure rates - Li, Y. H.; Schafer, W. D. (Paper presented at the annual meeting of the American Educational Research Association, 2004)
{Incomplete PDF file, 202 KB}

Contributions to the theory and practice of computerized adaptive testing - Theo Eggen (2004)

Detecting exposed test items in computer-based testing - Han, N.; Hambleton, R. K. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)
{PDF file, 1.245 MB}

Developing tailored instruments: Item banking and computerized adaptive assessment - Chang, C-H. (Paper presented at the conference “Advances in Health Outcomes Measurement: Exploring the Current State and the Future of Item Response Theory, 2004)
{PDF file, 181 KB}

Developing tailored instruments: Item banking and computerized adaptive assessment - Bjorner, J. B. (Paper presented at the conference “Advances in Health Outcomes Measurement: Exploring the Current State and the Future of Item Response Theory, 2004)
{PDF file, 406 KB}

The development and evaluation of a software prototype for computer-adaptive testing - Lilley, M; Barker, T; Britton, C (Computers and Education, 2004)
Keywords: computerized adaptive testing

Effects of practical constraints on item selection rules at the early stages of computerized adaptive testing - Chen, S-Y.; Ankenmann, R. D. (Journal of Educational Measurement, 2004)
The purpose of this study was to compare the effects of four item selection rules--(1) Fisher information (F), (2) Fisher information with a posterior distribution (FP), (3) Kullback-Leibler information with a posterior distribution (KP), and (4) completely randomized item selection (RN)--with respect to the precision of trait estimation and the extent of item usage at the early stages of computerized adaptive testing. The comparison of the four item selection rules was carried out under three conditions: (1) using only the item information function as the item selection criterion; (2) using both the item information function and content balancing; and (3) using the item information function, content balancing, and item exposure control. When test length was less than 10 items, FP and KP tended to outperform F at extreme trait levels in Condition 1. However, in more realistic settings, it could not be concluded that FP and KP outperformed F, especially when item exposure control was imposed. When test length was greater than 10 items, the three nonrandom item selection procedures performed similarly no matter what the condition was, while F had slightly higher item usage. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: computerized adaptive testing; item selection rules; practical constraints

Estimating ability and item-selection strategy in self-adapted testing: A latent class approach - Revuelta, J. (Journal of Educational and Behavioral Statistics, 2004)
This article presents a psychometric model for estimating ability and item-selection strategies in self-adapted testing. In contrast to computer adaptive testing, in self-adapted testing the examinees are allowed to select the difficulty of the items. The item-selection strategy is defined as the distribution of difficulty conditional on the responses given to previous items. The article shows that missing responses in self-adapted testing are missing at random and can be ignored in the estimation of ability. However, the item-selection strategy cannot always be ignored in such an estimation. An EM algorithm is presented to estimate an examinee's ability and strategies, and a model fit is evaluated using Akaike's information criterion. The article includes an application with real data to illustrate how the model can be used in practice for evaluating hypotheses, estimating ability, and identifying strategies. In the example, four strategies were identified and related to examinees' ability. It was shown that individual examinees tended not to follow a consistent strategy throughout the test. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: estimating ability; item-selection strategies; psychometric model; self-adapted testing

Evaluating scale stability of a computer adaptive testing system - Guo, F.; Wang, L. (GMAC, 2004)

Evaluating the effects of several multi-stage testing design variables on selected psychometric outcomes for certification and licensure assessment - Zenisky, A. L. (2004)

Évaluation et multimédia dans l'apprentissage d'une L2 [Assessment and multimedia in learning an L2] - Laurier, M. (ReCALL, 2004)
In the first part of this paper different areas where technology may be used for second language assessment are described. First, item banking operations, which are generally based on item Response Theory but not necessarily restricted to dichotomously scored items, facilitate assessment task organization and require technological support. Second, technology may help to design more authentic assessment tasks or may be needed in some direct testing situations. Third, the assessment environment may be more adapted and more stimulating when technology is used to give the student more control. The second part of the paper presents different functions of assessment. The monitoring function (often called formative assessment) aims at adapting the classroom activities to students and to provide continuous feedback. Technology may be used to train the teachers in monitoring techniques, to organize data or to produce diagnostic information; electronic portfolios or quizzes that are built in some educational software may also be used for monitoring. The placement function is probably the one in which the application of computer adaptive testing procedures (e.g. French CAPT) is the most appropriate. Automatic scoring devices may also be used for placement purposes. Finally the certification function requires more valid and more reliable tools. Technology may be used to enhance the testing situation (to make it more authentic) or to facilitate data processing during the construction of a test. Almond et al. (2002) propose a four component model (Selection, Presentation, Scoring and Response) for designing assessment systems. Each component must be planned taking into account the assessment function.
Keywords: Adaptive Testing; Computer Assisted Instruction; Educational; Foreign Language Learning; Program Evaluation; Technology computerized adaptive testing

Evaluation of the CATSIB DIF procedure in a pretest setting - Nandakumar, R.; Roussos, L. A. (Journal of Educational and Behavioral Statistics, 2004)
A new procedure, CATSIB, for assessing differential item functioning (DIF) on computerized adaptive tests (CATs) is proposed. CATSIB, a modified SIBTEST procedure, matches test takers on estimated ability and controls for impact-induced Type I error inflation by employing a CAT version of the SIBTEST "regression correction." The performance of CATSIB in terms of detection of DIF in pretest items was evaluated in a simulation study. Simulated test takers were adoptively administered 25 operational items from a pool of 1,000 and were linearly administered 16 pretest items that were evaluated for DIF. Sample size varied from 250 to 500 in each group. Simulated impact levels ranged from a 0- to 1-standard-deviation difference in mean ability levels. The results showed that CATSIB with the regression correction displayed good control over Type 1 error, whereas CATSIB without the regression correction displayed impact-induced Type 1 error inflation. With 500 test takers in each group, power rates were exceptionally high (84% to 99%) for values of DIF at the boundary between moderate and large DIF. For smaller samples of 250 test takers in each group, the corresponding power rates ranged from 47% to 95%. In addition, in all cases, CATSIB was very accurate in estimating the true values of DIF, displaying at most only minor estimation bias. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: computerized adaptive tests; differential item functioning

ffects of practical constraints on item selection rules at the early stages of computerized adaptive testing - Chen, Y.-Y.; Ankenmann, R. D. (Journal of Educational Measurement, 2004)

Impact of Test Design, Item Quality, and Item Bank Size on the Psychometric Properties of Computer-Based Credentialing Examinations - Xing, Dehui; Hambleton, Ronald K. (Educational and Psychological Measurement, 2004)
Computer-based testing by credentialing agencies has become common; however, selecting a test design is difficult because several good ones are available—parallel forms, computer adaptive (CAT), and multistage (MST). In this study, three computerbased test designs under some common examination conditions were investigated. Item bank size and item quality had a practically significant impact on decision consistency and accuracy. Even in nearly ideal situations, the choice of test design was not a factor in the results. Two conclusions follow from the findings: (a) More time and resources should be committed to expanding the size and quality of item banks, and (b) designs that individualize an exam administration such as MST and CAT may not be helpful when the primary purpose of the examination is to make pass-fail decisions and conditions are present for using parallel forms with a target information function that can be centered on the passing score.

Implementation and Measurement Efficiency of Multidimensional Computerized Adaptive Testing - Wang, Wen-Chung; Chen, Po-Hsi (Applied Psychological Measurement, 2004)
Multidimensional adaptive testing (MAT) procedures are proposed for the measurement of several latent traits by a single examination. Bayesian latent trait estimation and adaptive item selection are derived. Simulations were conducted to compare the measurement efficiency of MAT with those of unidimensional adaptive testing and random administration. The results showed that the higher the correlation between latent traits, the more latent traits there were, and the more scoring levels there were in the items, the more efficient MAT was than the other two procedures. For tests containing multidimensional items, only MAT is applicable, whereas unidimensional adaptive testing is not. Issues in implementing MAT are discussed.

Investigating the effects of selected multi-stage test design alternatives on credentialing outcomes - Zenisky, A. L.; Hambleton, R. K. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)
{PDF file, 129 KB}

An investigation of two combination procedures of SPRT for three-category classification decisions in computerized classification test - Jiao, H.; Wang, S; Lau, CA (annual meeting of the American Educational Research Association, 2004)
annual meeting of the American Educational Research Association, San Antonio
Keywords: computerized adaptive testing; Computerized classification testing; sequential probability ratio testing

An investigation of two combination procedures of SPRT for three-category decisions in computerized classification test - Jiao, H.; Wang, S; Lau, A (2004)
{PDF file, 649 KB}

Item parameter recovery with adaptive tests - Do, B.-R.; Chuah, S. C.; F Drasgow (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)
#DO04-01 {PDF file, 379 KB}

Kann die Konfundierung von Konzentrationsleistung und Aktivierung durch adaptives Testen mit dern FAKT vermieden werden? [Avoiding the confounding of concentration performance and activation by adaptive testing with the FACT] - Frey, A.; Moosbrugger, H. (Zeitschrift für Differentielle und Diagnostische Psychologie, 2004)
The study investigates the effect of computerized adaptive testing strategies on the confounding of concentration performance with activation. A sample of 54 participants was administered 1 out of 3 versions (2 adaptive, 1 non-adaptive) of the computerized Frankfurt Adaptive Concentration Test FACT (Moosbrugger & Heyden, 1997) at three subsequent points in time. During the test administration changes in activation (electrodermal activity) were recorded. The results pinpoint a confounding of concentration performance with activation for the non-adaptive test version, but not for the adaptive test versions (p = .01). Thus, adaptive FACT testing strategies can remove the confounding of concentration performance with activation, thereby increasing the discriminant validity. In conclusion, an attention-focusing-hypothesis is formulated to explain the observed effect. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Adaptive Testing; Computer Assisted Testing; Concentration; Performance; Testing computerized adaptive testing

A learning environment for english for academic purposes based on adaptive tests and task-based systems - PITON-GONÇALVES, J.; ALUISIO, S. M.; MENDONCA, L. H.; NOVAES, O. O. (Intelligent Tutoring Systems., 2004)

A Learning Environment for English for Academic Purposes Based on Adaptive Tests and Task-Based Systems - Gonçalves, Jean P.; Aluisio, Sandra M.; de Oliveira, Leandro H.M.; Oliveira Jr., Osvaldo N. (Intelligent Tutoring Systems, 2004)

Mokken Scale Analysis Using Hierarchical Clustering Procedures - van Abswoude, Alexandra A. H.; Vermunt, Jeroen K.; Hemker, Bas T.; van der Ark, L. Andries (Applied Psychological Measurement, 2004)
Mokken scale analysis (MSA) can be used to assess and build unidimensional scales from an item pool that is sensitive to multiple dimensions. These scales satisfy a set of scaling conditions, one of which follows from the model of monotone homogeneity. An important drawback of the MSA program is that the sequential item selection and scale construction procedure may not find the dominant underlying dimensionality of the responses to a set of items. The authors investigated alternative hierarchical item selection procedures and compared the performance of four hierarchical methods and the sequential clustering method in the MSA context. The results showed that hierarchical clustering methods can improve the search process of the dominant dimensionality of a data matrix. In particular, the complete linkage and scale linkage methods were promising in finding the dimensionality of the item response data from a set of items.

Mutual information item selection in multiple-category classification CAT - Weissman, A. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)
#WE04-02

New methods for CBT item pool evaluation - Wang, L. (Paper presented at the annual meeting of the American Educational Research Association, 2004)
#WA04-02 {PDF file, 1.005 MB}

Optimal testing with easy items in computerized adaptive testing (Measurement and Research Department Report 2004-2) - Theo Eggen; Verschoor, A. J. (2004)

Practical methods for dealing with 'not applicable' item responses in the AMC Linear Disability Score project - Holman, R.; Glas, C. A.; Lindeboom, R.; Zwinderman, A. H.; de Haan, R. J. (Health and Quality of Life Outcomes, 2004)
BACKGROUND: Whenever questionnaires are used to collect data on constructs, such as functional status or health related quality of life, it is unlikely that all respondents will respond to all items. This paper examines ways of dealing with responses in a 'not applicable' category to items included in the AMC Linear Disability Score (ALDS) project item bank. METHODS: The data examined in this paper come from the responses of 392 respondents to 32 items and form part of the calibration sample for the ALDS item bank. The data are analysed using the one-parameter logistic item response theory model. The four practical strategies for dealing with this type of response are: cold deck imputation; hot deck imputation; treating the missing responses as if these items had never been offered to those individual patients; and using a model which takes account of the 'tendency to respond to items'. RESULTS: The item and respondent population parameter estimates were very similar for the strategies involving hot deck imputation; treating the missing responses as if these items had never been offered to those individual patients; and using a model which takes account of the 'tendency to respond to items'. The estimates obtained using the cold deck imputation method were substantially different. CONCLUSIONS: The cold deck imputation method was not considered suitable for use in the ALDS item bank. The other three methods described can be usefully implemented in the ALDS item bank, depending on the purpose of the data analysis to be carried out. These three methods may be useful for other data sets examining similar constructs, when item response theory based methods are used.
Holman, RebeccaGlas, Cees A WLindeboom, RobertZwinderman, Aeilko Hde Haan, Rob JEnglandHealth Qual Life Outcomes. 2004 Jun 16;2:29.
Keywords: *Disability Evaluation; *Health Surveys; *Logistic Models; *Questionnaires; Activities of Daily Living/*classification; Data Interpretation, Statistical; Health Status; Humans; Pilot Projects; Probability; Quality of Life; Severity of Illness Index

Pre-equating: a simulation study based on a large scale assessment model - Taherbhai, H. M.; Young, M. J. (Journal of Applied Measurement, 2004)
Although post-equating (PE) has proven to be an acceptable method in the scaling and equating of items and forms, there are times when the turn-around period for equating and converting raw scores to scale scores is so small that PE cannot be undertaken within the prescribed time frame. In such cases, pre-equating (PrE) could be considered as an acceptable alternative. Assessing the feasibility of using item calibrations from the item bank (as in PrE) is conditioned on the equivalency of the calibrations and the errors associated with it vis a vis the results obtained via PE. This paper creates item banks over three periods of item introduction into the banks and uses the Rasch model in examining data with respect to the recovery of item parameters, the measurement error, and the effect cut-points have on examinee placement in both the PrE and PE situations. Results indicate that PrE is a viable solution to PE provided the stability of the item calibrations are enhanced by using large sample sizes (perhaps as large as full-population) in populating the item bank.
1529-7713Journal Article
Keywords: *Databases; *Models, Theoretical; Calibration; Human; Psychometrics; Reference Values; Reproducibility of Results

Protecting the integrity of computer-adaptive licensure tests: Results of a legal challenge - Cizek, G. J. (Paper presented at the annual meeting of the American Educational Research Association, 2004)
{PDF file, 191 KB}

Refining the conceptual basis for rehabilitation outcome measurement: personal care and instrumental activities domain - Coster, W. J.; Haley, S. M.; Andres, P. L.; Ludlow, L. H.; Bond, T. L.; Ni, P. S. (Medical Care, 2004)
BACKGROUND: Rehabilitation outcome measures routinely include content on performance of daily activities; however, the conceptual basis for item selection is rarely specified. These instruments differ significantly in format, number, and specificity of daily activity items and in the measurement dimensions and type of scale used to specify levels of performance. We propose that a requirement for upper limb and hand skills underlies many activities of daily living (ADL) and instrumental activities of daily living (IADL) items in current instruments, and that items selected based on this definition can be placed along a single functional continuum. OBJECTIVE: To examine the dimensional structure and content coverage of a Personal Care and Instrumental Activities item set and to examine the comparability of items from existing instruments and a set of new items as measures of this domain. METHODS: Participants (N = 477) from 3 different disability groups and 4 settings representing the continuum of postacute rehabilitation care were administered the newly developed Activity Measure for Post-Acute Care (AM-PAC), the SF-8, and an additional setting-specific measure: FIM (in-patient rehabilitation); MDS (skilled nursing facility); MDS-PAC (postacute settings); OASIS (home care); or PF-10 (outpatient clinic). Rasch (partial-credit model) analyses were conducted on a set of 62 items covering the Personal Care and Instrumental domain to examine item fit, item functioning, and category difficulty estimates and unidimensionality. RESULTS: After removing 6 misfitting items, the remaining 56 items fit acceptably along the hypothesized continuum. Analyses yielded different difficulty estimates for the maximum score (eg, "Independent performance") for items with comparable content from different instruments. Items showed little differential item functioning across age, diagnosis, or severity groups, and 92% of the participants fit the model. CONCLUSIONS: ADL and IADL items from existing rehabilitation outcomes instruments that depend on skilled upper limb and hand use can be located along a single continuum, along with the new personal care and instrumental items of the AM-PAC addressing gaps in content. Results support the validity of the proposed definition of the Personal Care and Instrumental Activities dimension of function as a guide for future development of rehabilitation outcome instruments, such as linked, setting-specific short forms and computerized adaptive testing approaches.
0025-7079Journal Article
Keywords: *Self Efficacy; *Sickness Impact Profile; Activities of Daily Living/*classification/psychology; Adult; Aged; Aged, 80 and over; Disability Evaluation; Factor Analysis, Statistical; Female; Humans; Male; Middle Aged; Outcome Assessment (Health Care)/*methods/statistics & numerical data; Questionnaires/*standards; Recovery of Function/physiology; Rehabilitation/*standards/statistics & numerical data; Reproducibility of Results; Research Support, U.S. Gov't, Non-P.H.S.; Research Support, U.S. Gov't, P.H.S.; Sensitivity and Specificity

Score comparability of short forms and computerized adaptive testing: Simulation study with the activity measure for post-acute care - Haley, S. M.; Coster, W. J.; Andres, P. L.; Kosinski, M.; Ni, P. (Archives of Physical Medicine and Rehabilitation, 2004)
OBJECTIVE: To compare simulated short-form and computerized adaptive testing (CAT) scores to scores obtained from complete item sets for each of the 3 domains of the Activity Measure for Post-Acute Care (AM-PAC). DESIGN: Prospective study. SETTING: Six postacute health care networks in the greater Boston metropolitan area, including inpatient acute rehabilitation, transitional care units, home care, and outpatient services. PARTICIPANTS: A convenience sample of 485 adult volunteers who were receiving skilled rehabilitation services. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Inpatient and community-based short forms and CAT applications were developed for each of 3 activity domains (physical & mobility, personal care & instrumental, applied cognition) using item pools constructed from new items and items from existing postacute care instruments. RESULTS: Simulated CAT scores correlated highly with score estimates from the total item pool in each domain (4- and 6-item CAT r range,.90-.95; 10-item CAT r range,.96-.98). Scores on the 10-item short forms constructed for inpatient and community settings also provided good estimates of the AM-PAC item pool scores for the physical & movement and personal care & instrumental domains, but were less consistent in the applied cognition domain. Confidence intervals around individual scores were greater in the short forms than for the CATs. CONCLUSIONS: Accurate scoring estimates for AM-PAC domains can be obtained with either the setting-specific short forms or the CATs. The strong relationship between CAT and item pool scores can be attributed to the CAT's ability to select specific items to match individual responses. The CAT may have additional advantages over short forms in practicality, efficiency, and the potential for providing more precise scoring estimates for individuals.
Haley, Stephen MCoster, Wendy JAndres, Patricia LKosinski, MarkNi, PengshengR01 hd43568/hd/nichdComparative StudyMulticenter StudyResearch Support, U.S. Gov't, Non-P.H.S.Research Support, U.S. Gov't, P.H.S.United StatesArchives of physical medicine and rehabilitationArch Phys Med Rehabil. 2004 Apr;85(4):661-6.
Keywords: Boston; Factor Analysis, Statistical; Humans; Outcome Assessment (Health Care)/*methods; Prospective Studies; Questionnaires/standards; Rehabilitation/*standards; Subacute Care/*standards

A sequential Bayesian procedure for item calibration in multistage testing - van der Linden, W. J.; Alan D Mead (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)

Sequential estimation in variable length computerized adaptive testing - Chang, I. Y. (Journal of Statistical Planning and Inference, 2004)
With the advent of modern computer technology, there have been growing e3orts in recent years to computerize standardized tests, including the popular Graduate Record Examination (GRE), the Graduate Management Admission Test (GMAT) and the Test of English as a Foreign Language (TOEFL). Many of such computer-based tests are known as the computerized adaptive tests, a major feature of which is that, depending on their performance in the course of testing, di3erent examinees may be given with di3erent sets of items (questions). In doing so, items can be e>ciently utilized to yield maximum accuracy for estimation of examinees’ ability traits. We consider, in this article, one type of such tests where test lengths vary with examinees to yield approximately same predetermined accuracy for all ability traits. A comprehensive large sample theory is developed for the expected test length and the sequential point and interval estimates of the latent trait. Extensive simulations are conducted with results showing that the large sample approximations are adequate for realistic sample sizes.

A sharing item response theory model for computerized adaptive testing - Segall, D. O. (Journal of Educational and Behavioral Statistics, 2004)
A new sharing item response theory (SIRT) model is presented which explicitly models the effects of sharing item content between informants and testtakers. This model is used to construct adaptive item selection and scoring rules that provide increased precision and reduced score gains in instances where sharing occurs. The adaptive item selection rules are expressed as functions of the item’s exposure rate in addition to other commonly used properties (characterized by difficulty, discrimination, and guessing parameters). Based on the results of simulated item responses, the new item selection and scoring algorithms compare favorably to the Sympson-Hetter exposure control method. The new SIRT approach provides higher reliability and lower score gains in instances where sharing occurs.
References .American Educational Research Assn, US

Siette: a web-based tool for adaptive testing - Conejo, R; Guzmán, E; Millán, E; Trella, M; Pérez-De-La-Cruz, JL; Ríos, A (International Journal of Artificial Intelligence in Education, 2004)
Keywords: computerized adaptive testing

State-of-the-art and adaptive open-closed items in adaptive foreign language assessment - Giouroglou, H.; Economides, A. A. (2004)

Statistics for detecting disclosed items in a CAT environment - Lu, Y.,; Hambleton, R. K. (Metodologia de Las Ciencias del Comportamiento., 2004)

Strategies for Controlling Item Exposure in Computerized Adaptive Testing With the Generalized Partial Credit Model - Davis, Laurie Laughlin (Applied Psychological Measurement, 2004)
Choosing a strategy for controlling item exposure has become an integral part of test development for computerized adaptive testing (CAT). This study investigated the performance of six procedures for controlling item exposure in a series of simulated CATs under the generalized partial credit model. In addition to a no-exposure control baseline condition, the randomesque, modified-within-.10-logits, Sympson-Hetter, conditional Sympson-Hetter, a-stratified with multiple-stratification, and enhanced a-stratified with multiple-stratification procedures were implemented to control exposure rates. Two variations of the randomesque and modified-within-.10-logits procedures were examined, which varied the size of the item group from which the next item to be administered was randomly selected. The results indicate that although the conditional Sympson-Hetter provides somewhat lower maximum exposure rates, the randomesque and modified-within-.10-logits procedures with the six-item group variation have great utility for controlling overlap rates and increasing pool utilization and should be given further consideration.

Strategies for controlling item exposure in computerized adaptive testing with the generalized partial credit model - Davis, L. L. (Applied Psychological Measurement, 2004)
Choosing a strategy for controlling item exposure has become an integral part of test development for computerized adaptive testing (CAT). This study investigated the performance of six procedures for controlling item exposure in a series of simulated CATs under the generalized partial credit model. In addition to a no-exposure control baseline condition, the randomesque, modified-within-.10-logits, Sympson-Hetter, conditional Sympson-Hetter, a-stratified with multiple-stratification, and enhanced a-stratified with multiple-stratification procedures were implemented to control exposure rates. Two variations of the randomesque and modified-within-.10-logits procedures were examined, which varied the size of the item group from which the next item to be administered was randomly selected. The results indicate that although the conditional Sympson-Hetter provides somewhat lower maximum exposure rates, the randomesque and modified-within-.10-logits procedures with the six-item group variation have great utility for controlling overlap rates and increasing pool utilization and should be given further consideration. (PsycINFO Database Record (c) 2007 APA, all rights reserved)
Keywords: computerized adaptive testing; generalized partial credit model; item exposure

Strategies for controlling testlet exposure rates in computerized adaptive testing systems - Boyd, Aimee Michelle (Dissertation Abstracts International: Section B: The Sciences & Engineering, 2004)
Exposure control procedures in computerized adaptive testing (CAT) systems protect item pools from being compromised, however, this impacts measurement precision. Previous research indicates that exposure control procedures perform differently for dichotomously scored versus polytomously scored CAT systems. For dichotomously scored CATs, conditional selection procedures are often the optimal choice, while randomization procedures perform best for polytomously scored CATs. CAT systems modeled with testlet response theory have not been examined to determine optimal exposure control procedures. This dissertation examined various exposure control procedures in testlet-based CAT systems using the three-parameter logistic testlet response theory model and the partial credit model. The exposure control procedures were the randomesque procedure, the modified within .10 logits procedure, two levels of the progressive restricted procedure, and two levels of the Sympson-Hetter procedure. Each of these was compared to a baseline no exposure control procedure, maximum information. The testlets were reading passages with six to ten multiple-choice items. The CAT systems consisted of maximum information testlet selection contingent on an exposure control procedure and content balancing for passage type and the number of items per passage; expected a posteriori ability estimation; and a fixed length stopping rule of seven testlets totaling fifty multiple-choice items. Measurement precision and exposure rates were examined to evaluate the effectiveness of the exposure control procedures for each measurement model. The exposure control procedures yielded similar results for measurement precision within the models. The exposure rates distinguished which exposure control procedures were most effective. The Sympson-Hetter conditions, which are conditional procedures, maintained the pre-specified maximum exposure rate, but performed very poorly in terms of pool utilization. The randomization procedures, randomesque and modified within .10 logits, yielded low maximum exposure rates, but used only about 70% of the testlet pool. Surprisingly, the progressive restricted procedure, which is a combination of both a conditional and randomization procedure, yielded the best results in its ability to maintain and control the maximum exposure rate and it used the entire testlet pool. The progressive restricted conditions were the optimal procedures for both the partial credit CAT systems and the three-parameter logistic testlet response theory CAT systems. (PsycINFO Database Record (c) 2004 APA, all rights reserved).

A study of multiple stage adaptive test designs - Armstrong, R. D.; Edmonds, J. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2004)
{PDF file, 288 KB}

Test difficulty and stereotype threat on the GRE General Test - Stricker, L. J.,; Bejar, I. I. (Journal of Applied Social Psychology, 2004)

Testing vocabulary knowledge: Size, strength, and computer adaptiveness - Laufer, B.; Goldstein, Z. (Language Learning, 2004)
(from the journal abstract) In this article, we describe the development and trial of a bilingual computerized test of vocabulary size, the number of words the learner knows, and strength, a combination of four aspects of knowledge of meaning that are assumed to constitute a hierarchy of difficulty: passive recognition (easiest), active recognition, passive recall, and active recall (hardest). The participants were 435 learners of English as a second language. We investigated whether the above hierarchy was valid and which strength modality correlated best with classroom language performance. Results showed that the hypothesized hierarchy was present at all word frequency levels, that passive recall was the best predictor of classroom language performance, and that growth in vocabulary knowledge was different for the different strength modalities. (PsycINFO Database Record (c) 2004 APA, all rights reserved).
References .Blackwell Publishing, United Kingdom

Understanding computerized adaptive testing: From Robbins-Munro to Lord and beyond - Chang, Hua-Hua (2004)

Using patterns of summed scores in paper-and-pencil tests and computer-adaptive tests to detect misfitting item score patterns - Meijer, R. R. (Journal of Educational Measurement, 2004)
Two new methods have been proposed to determine unexpected sum scores on subtests (testlets) both for paper-and-pencil tests and computer adaptive tests. A method based on a conservative bound using the hypergeometric distribution, denoted ρ, was compared with a method where the probability for each score combination was calculated using a highest density region (HDR). Furthermore, these methods were compared with the standardized log-likelihood statistic with and without a correction for the estimated latent trait value (denoted as l-super(*)-sub(z) and l-sub(z), respectively). Data were simulated on the basis of the one-parameter logistic model, and both parametric and nonparametric logistic regression was used to obtain estimates of the latent trait. Results showed that it is important to take the trait level into account when comparing subtest scores. In a nonparametric item response theory (IRT) context, on adapted version of the HDR method was a powerful alterative to ρ. In a parametric IRT context, results showed that l-super(*)-sub(z) had the highest power when the data were simulated conditionally on the estimated latent trait level. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Computer Assisted Testing; Item Response Theory; person Fit; Test Scores

Using Set Covering with Item Sampling to Analyze the Infeasibility of Linear Programming Test Assembly Models - Huitzing, Hiddo A. (Applied Psychological Measurement, 2004)
This article shows how set covering with item sampling (SCIS) methods can be used in the analysis and preanalysis of linear programming models for test assembly (LPTA). LPTA models can construct tests, fulfilling a set of constraints set by the test assembler. Sometimes, no solution to the LPTA model exists. The model is then said to be infeasible. Causes of infeasibility can be difficult to find. A method is proposed that constitutes a helpful tool for test assemblers to detect infeasibility before hand and, in the case of infeasibility, give insight into its causes. This method is based on SCIS. Although SCIS can help to detect feasibility or infeasibility, its power lies in pinpointing causes of infeasibility such as irreducible infeasible sets of constraints. Methods to resolve infeasibility are also given, minimizing the model deviations. A simulation study is presented, offering a guide to test assemblers to analyze and solve infeasibility.

Validating the German computerized adaptive test for anxiety on healthy sample (A-CAT) - Becker, J.; Walter, O. B.; Fliege, H.; Bjorner, J. B.; Kocalevent, R. D.; Schmid, G.; Klapp, B. F.; Rose, M. (Quality of Life Research, 2004)

Accuracy of reading and mathematics ability estimates under the shadow-test constraint MCAT - Li, Y. H.; Schafer, W. D. (Paper presented at the annual meeting of the American Educational Research Association, 2003)

An adaptation of stochastic curtailment to truncate Wald’s SPRT in computerized adaptive testing - Finkelman, M. (National Center for Research on Evaluation, Standards, and Student Testing, 2003)
Computerized adaptive testing (CAT) has been shown to increase eÆciency in educational measurement. One common application of CAT is to classify students as either pro cient or not proficient in ability. A truncated form of Wald's sequential probability ratio test (SPRT), in which examination is halted after a prespeci ed number of questions, has been proposed to provide a diagnosis of prociency. This article studies the further truncation provided by stochastic curtailment, where an exam is stopped early if completion of the remaining questions would be unlikely to alter the classi cation of the examinee. In a simulation study presented, the increased truncation is shown to offer substantial improvement in test length with only a slight decrease in accuracy.

Adaptive exploration of assessment results under uncertainty - Lamboudis, D.; Economides, A. A.; Papastergiou, A. (2003)

An adaptive exposure control algorithm for computerized adaptive testing using a sharing item response theory model - Segall, D. O. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 191 KB}

Alpha-stratified adaptive testing with large numbers of content constraints - van der Linden, W. J.; Chang, Hua-Hua (Applied Psychological Measurement, 2003)

The assembly of multiple form structures - Armstrong, R. D.; Little, J. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 418 KB}

The assembly of multiple stage adaptive tests with discrete items - Armstrong, R. D.; Edmonds, J.J. (2003)

Assessing CAT security breaches by the item pooling index - Chang, Hua-Hua; Zhang, J. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)

Assessing question banks - Bull, J.; Dalziel, J.; Vreeland, T. (Reusing online resources: A sustanable approach to e-learning, 2003)
In Chapter 14, Joanna Bull and James Daziel provide a comprehensive treatment of the issues surrounding the use of Question Banks and Computer Assisted Assessment, and provide a number of excellent examples of implementations. In their review of the technologies employed in Computer Assisted Assessment the authors include Computer Adaptive Testing and data generation. The authors reveal significant issues involving the impact of Intellectual Property rights and computer assisted assessment and make important suggestions for strategies to overcome these obstacles. (PsycINFO Database Record (c) 2005 APA )http://www-jime.open.ac.uk/2003/1/ (journal abstract)
Keywords: Computer Assisted Testing; Curriculum Based Assessment; Education; Technology computerized adaptive testing

Assessing the efficiency of item selection in computerized adaptive testing - Weissman, A. (Paper presented at the annual meeting of the American Educational Research Association, 2003)
{PDF file, 96 KB}

a-Stratified multistage CAT design with content-blocking - Yi, Q.; Chang, H.-H. (British Journal of Mathematical and Statistical Psychology, 2003)

Bayesian checks on outlying response times in computerized adaptive testing - van der Linden, W. J. (2003)

A Bayesian method for the detection of item preknowledge in computerized adaptive testing - McLeod, L.; Lewis, C.; Thissen, D. (Applied Psychological Measurement, 2003)
With the increased use of continuous testing in computerized adaptive testing, new concerns about test security have evolved, such as how to ensure that items in an item pool are safeguarded from theft. In this article, procedures to detect test takers using item preknowledge are explored. When test takers use item preknowledge, their item responses deviate from the underlying item response theory (IRT) model, and estimated abilities may be inflated. This deviation may be detected through the use of person-fit indices. A Bayesian posterior log odds ratio index is proposed for detecting the use of item preknowledge. In this approach to person fit, the estimated probability that each test taker has preknowledge of items is updated after each item response. These probabilities are based on the IRT parameters, a model specifying the probability that each item has been memorized, and the test taker's item responses. Simulations based on an operational computerized adaptive test (CAT) pool are used to demonstrate the use of the odds ratio index. (PsycINFO Database Record (c) 2005 APA )
Keywords: Adaptive Testing; Cheating; Computer Assisted Testing; Individual Differences computerized adaptive testing; Item; Item Analysis (Statistical); Mathematical Modeling; Response Theory

A Bayesian method for the detection of item preknowledge in computerized adaptive testing - McLeod L. D., Lewis, C.,; Thissen, D. (Applied Psychological Measurement, 2003)

Calibrating CAT item pools and online pretest items using MCMC methods - Segall, D. O. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 155 KB}

Calibrating CAT pools and online pretest items using marginal maximum likelihood methods - Pommerich, M; Segall, D. O. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 284 KB}

Calibrating CAT pools and online pretest items using nonparametric and adjusted marginal maximum likelihood methods - Krass, I. A.; Williams, B. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
PDF file, 128 K

Calibration of an item pool for assessing the burden of headaches: an application of item response theory to the Headache Impact Test (HIT) - Bjorner, J. B.; Kosinski, M.; Ware, J. E., Jr. (Quality of Life Research, 2003)
BACKGROUND: Measurement of headache impact is important in clinical trials, case detection, and the clinical monitoring of patients. Computerized adaptive testing (CAT) of headache impact has potential advantages over traditional fixed-length tests in terms of precision, relevance, real-time quality control and flexibility. OBJECTIVE: To develop an item pool that can be used for a computerized adaptive test of headache impact. METHODS: We analyzed responses to four well-known tests of headache impact from a population-based sample of recent headache sufferers (n = 1016). We used confirmatory factor analysis for categorical data and analyses based on item response theory (IRT). RESULTS: In factor analyses, we found very high correlations between the factors hypothesized by the original test constructers, both within and between the original questionnaires. These results suggest that a single score of headache impact is sufficient. We established a pool of 47 items which fitted the generalized partial credit IRT model. By simulating a computerized adaptive health test we showed that an adaptive test of only five items had a very high concordance with the score based on all items and that different worst-case item selection scenarios did not lead to bias. CONCLUSION: We have established a headache impact item pool that can be used in CAT of headache impact.
0962-9343Journal Article
Keywords: *Cost of Illness; *Decision Support Techniques; *Sickness Impact Profile; Adolescent; Adult; Aged; Comparative Study; Disability Evaluation; Factor Analysis, Statistical; Headache/*psychology; Health Surveys; Human; Longitudinal Studies; Middle Aged; Migraine/psychology; Models, Psychological; Psychometrics/*methods; Quality of Life/*psychology; Software; Support, Non-U.S. Gov't

Can an item response theory-based pain item bank enhance measurement precision? - Lai, J-S.; Dineen, K.; Cella, D.; Von Roenn, J. (Clinical Therapeutics, 2003)

Can We Assess Pre-K Kids With Computer-Based Tests: STAR Early Literacy Data - J. R. McBride (Presentation to the 33rd Annual National Conference on Large-Scale Assessment., 2003)

CAT-ASVAB prototype Internet delivery system: Final report (FR-03-06) - Sticha, P. J.; Barber, G. (2003)
{PDF file, 393 KB}

Cognitive CAT in foreign language assessment - Giouroglou, H.; Economides, A. A. (Proceedings 11th International PEG Conference, 2003)

A comparative study of item exposure control methods in computerized adaptive testing - Chang, S-W.; Ansley, T. N. (Journal of Educational Measurement, 2003)
This study compared the properties of five methods of item exposure control within the purview of estimating examinees' abilities in a computerized adaptive testing (CAT) context. Each exposure control algorithm was incorporated into the item selection procedure and the adaptive testing progressed based on the CAT design established for this study. The merits and shortcomings of these strategies were considered under different item pool sizes and different desired maximum exposure rates and were evaluated in light of the observed maximum exposure rates, the test overlap rates, and the conditional standard errors of measurement. Each method had its advantages and disadvantages, but no one possessed all of the desired characteristics. There was a clear and logical trade-off between item exposure control and measurement precision. The M. L. Stocking and C. Lewis conditional multinomial procedure and, to a slightly lesser extent, the T. Davey and C. G. Parshall method seemed to be the most promising considering all of the factors that this study addressed. (PsycINFO Database Record (c) 2005 APA )
Keywords: Adaptive Testing; Computer Assisted Testing; Educational; Item Analysis (Statistical); Measurement; Strategies computerized adaptive testing

A comparison of exposure control procedures in CAT systems based on different measurement models for testlets using the verbal reasoning section of the MCAT - Boyd, A. M; Dodd, B. G.; Fitzpatrick, S. J. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 405 KB}

A comparison of item exposure control procedures using a CAT system based on the generalized partial credit model - Burt, W. M; Kim, S.-J; Davis, L. L.; Dodd, B. G. (Paper presented at the annual meeting of the American Educational Research Association, 2003)
{PDF file, 265 KB}

A comparison of learning potential results at various educational levels - De Beer, M. (Paper presented at the 6th Annual Society for Industrial and Organisational Psychology of South Africa (SIOPSA) conference, 2003)
{PDF file, 391 KB}

Comparison of multi-stage tests with computer adaptive and paper and pencil tests - Rotou, O.; Patsula, L.; Steffen, M.; Rizavi, S. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 695 KB}

A computer adaptive testing simulation applied to the FIM instrument motor component - Dijkers, M.P. (Arch Phys Med Rehabil, 2003)

Computer-adaptive test for measuring personality factors using item response theory - Macdonald, Paul Lawrence (Dissertation Abstracts International: Section B: The Sciences & Engineering, 2003)
The aim of the present research was to develop a computer adaptive test with the graded response model to measure the Five Factor Model of personality attributes. In the first of three studies, simulated items and simulated examinees were used to investigate systematically the impact of several variables on the accuracy and efficiency of a computer adaptive test. Item test banks containing more items, items with greater trait discrimination, and more response options resulted in increased accuracy and efficiency of the computer adaptive test. It was also found that large stopping rule values required fewer items before stopping but had less accuracy compared to smaller stopping rule values. This demonstrated a trade-off between accuracy and efficiency such that greater measurement accuracy can be obtained at a cost of decreased test efficiency. In the second study, the archival responses of 501 participants to five 30-item test banks measuring the Five Factor Model of personality were utilized in simulations of a computer adaptive personality test. The computer adaptive test estimates of participant trait scores were highly correlated with the item response theory trait estimates, and the magnitude of the correlation was related directly to the stopping rule value with higher correlations and less measurement error being associated with smaller stopping rule values. It was also noted that the performance of the computer adaptive test was dependent on the personality factor being measured whereby Conscientiousness required the most number of items to be administered and Neuroticism required the least. The results confirmed that a simulated computer adaptive test using archival personality data could accurately and efficiently attain trait estimates. In the third study, 276 student participants selected response options with a click of a mouse in a computer adaptive personality test (CAPT) measuring the Big Five factors of the Five Factor Model of personality structure. Participant responses to alternative measures of the Big Five were also collected using conventional paper-and-pencil personality questionnaires. It was found that the CAPT obtained trait estimates that were very accurate even with very few administered items. Similarly, the CAPT trait estimates demonstrated moderate to high concurrent validity with the alternative Big Five measures, and the strength of the estimates varied as a result of the similarity of the personality items and assessment methodology. It was also found that the computer adaptive test was accurately able to detect, with relatively few items, the relations between the measured personality traits and several socially interesting variables such as smoking behavior, alcohol consumption rating, and number of dates per month. Implications of the results of this research are discussed in terms of the utility of computer adaptive testing of personality characteristics. As well, methodological limitations of the studies are noted and directions for future research are considered. (PsycINFO Database Record (c) 2004 APA, all rights reserved).

Computerized adaptive rating scales for measuring managerial performance - Schneider, R. J.; Goff, M.; Anderson, S.; Borman, W. C. (International Journal of Selection and Assessment, 2003)
Computerized adaptive rating scales (CARS) had been developed to measure contextual or citizenship performance. This rating format used a paired-comparison protocol, presenting pairs of behavioral statements scaled according to effectiveness levels, and an iterative item response theory algorithm to obtain estimates of ratees' citizenship performance (W. C. Borman et al, 2001). In the present research, we developed CARS to measure the entire managerial performance domain, including task and citizenship performance, thus addressing a major limitation of the earlier CARS. The paper describes this development effort, including an adjustment to the algorithm that reduces substantially the number of item pairs required to obtain almost as much precision in the performance estimates. (PsycINFO Database Record (c) 2005 APA )
Keywords: Adaptive Testing; Algorithms; Associations; Citizenship; Computer Assisted Testing; Construction; Contextual; Item Response Theory; Job Performance; Management; Management Personnel; Rating Scales; Test

Computerized adaptive testing - Ponsoda, V.; Olea, J. (2003)

Computerized adaptive testing: A comparison of three content balancing methods - Leung, C-K..; Chang, Hua-Hua; Hau, K-T.; Wen. Z. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 227 KB}

Computerized adaptive testing: A comparison of three content balancing methods - Leung, C-K..; Chang, Hua-Hua; Hau, K-T. (The Journal of Technology, Learning and Assessment, 2003)
Content balancing is often a practical consideration in the design of computerized adaptive testing (CAT). This study compared three content balancing methods, namely, the constrained CAT (CCAT), the modified constrained CAT (MCCAT), and the modified multinomial model (MMM), under various conditions of test length and target maximum exposure rate. Results of a series of simulation studies indicate that there is no systematic effect of content balancing method in measurement efficiency and pool utilization. However, among the three methods, the MMM appears to consistently over-expose fewer items.

Computerized adaptive testing using the nearest-neighbors criterion - Cheng, P. E.; Liou, M. (Applied Psychological Measurement, 2003)

Computerized adaptive testing using the nearest-neighbors criterion - Cheng, P. E.; Liou, M. (Applied Psychological Measurement, 2003)
Item selection procedures designed for computerized adaptive testing need to accurately estimate every taker's trait level (θ) and, at the same time, effectively use all items in a bank. Empirical studies showed that classical item selection procedures based on maximizing Fisher or other related information yielded highly varied item exposure rates; with these procedures, some items were frequently used whereas others were rarely selected. In the literature, methods have been proposed for controlling exposure rates; they tend to affect the accuracy in θ estimates, however. A modified version of the maximum Fisher information (MFI) criterion, coined the nearest neighbors (NN) criterion, is proposed in this study. The NN procedure improves to a moderate extent the undesirable item exposure rates associated with the MFI criterion and keeps sufficient precision in estimates. The NN criterion will be compared with a few other existing methods in an empirical study using the mean squared errors in θ estimates and plots of item exposure rates associated with different distributions. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: (Statistical); Adaptive Testing; Computer Assisted Testing; Item Analysis; Item Response Theory; Statistical Analysis; Statistical Estimation computerized adaptive testing; Statistical Tests

Computerized adaptive testing with item cloning - Glas, C. A. W.; van der Linden, W. J. (Applied Psychological Measurement, 2003)
(from the journal abstract) To increase the number of items available for adaptive testing and reduce the cost of item writing, the use of techniques of item cloning has been proposed. An important consequence of item cloning is possible variability between the item parameters. To deal with this variability, a multilevel item response (IRT) model is presented which allows for differences between the distributions of item parameters of families of item clones. A marginal maximum likelihood and a Bayesian procedure for estimating the hyperparameters are presented. In addition, an item-selection procedure for computerized adaptive testing with item cloning is presented which has the following two stages: First, a family of item clones is selected to be optimal at the estimate of the person parameter. Second, an item is randomly selected from the family for administration. Results from simulation studies based on an item pool from the Law School Admission Test (LSAT) illustrate the accuracy of these item pool calibration and adaptive testing procedures. (PsycINFO Database Record (c) 2003 APA, all rights reserved).
References .Sage Publications, US
Keywords: computerized adaptive testing

Constraining item exposure in computerized adaptive testing with shadow tests - van der Linden, W. J.; Veldkamp, B. P. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
#vdLI03-02

Constructing rotating item pools for constrained adaptive testing - Ariel, A.; Veldkamp, B.; van der Linden, W. J. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 395 KB}

Controlling item exposure and item eligibility in computerized adaptive testing - van der Linden, W. J.; Veldkamp, B. P. (2003)

Criterion item characteristic curve function for evaluating the differential weight procedure adjusted to on-line item calibration - Samejima, F. (Paper presented at the annual meeting of the NCME, 2003)

Developing an initial physical function item bank from existing sources - Bode, R. K.; Cella, D.; Lai, J. S.; Heinemann, A. W. (Journal of Applied Measurement, 2003)
The objective of this article is to illustrate incremental item banking using health-related quality of life data collected from two samples of patients receiving cancer treatment. The kinds of decisions one faces in establishing an item bank for computerized adaptive testing are also illustrated. Pre-calibration procedures include: identifying common items across databases; creating a new database with data from each pool; reverse-scoring "negative" items; identifying rating scales used in items; identifying pivot points in each rating scale; pivot anchoring items at comparable rating scale categories; and identifying items in each instrument that measure the construct of interest. A series of calibrations were conducted in which a small proportion of new items were added to the common core and misfitting items were identified and deleted until an initial item bank has been developed.
1529-7713Journal Article
Keywords: *Databases; *Sickness Impact Profile; Adaptation, Psychological; Data Collection; Humans; Neoplasms/*physiopathology/psychology/therapy; Psychometrics; Quality of Life/*psychology; Research Support, U.S. Gov't, P.H.S.; United States

Development and psychometric evaluation of the Flexilevel Scale of Shoulder Function (FLEX-SF) - Cook, K. F.; Roddey, T. S.; Gartsman, G M; Olson, S L (Medical Care (in press), 2003)
#CO03-01

Development of the Learning Potential Computerised Adaptive Test (LPCAT) - De Beer, M. (Unpublished manuscript., 2003)
{PDF file, 563 KB}

Development, reliability, and validity of a computerized adaptive version of the Schedule for Nonadaptive and Adaptive Personality - Simms, L. J. (Dissertation Abstracts International: Section B: The Sciences & Engineering, 2003)
Computerized adaptive testing (CAT) and Item Response Theory (IRT) techniques were applied to the Schedule for Nonadaptive and Adaptive Personality (SNAP) to create a more efficient measure with little or no cost to test reliability or validity. The SNAP includes 15 factor analytically derived and relatively unidimensional traits relevant to personality disorder. IRT item parameters were calibrated on item responses from a sample of 3,995 participants who completed the traditional paper-and-pencil (P&P) SNAP in a variety of university, community, and patient settings. Computerized simulations were conducted to test various adaptive testing algorithms, and the results informed the construction of the CAT version of the SNAP (SNAP-CAT). A validation study of the SNAP-CAT was conducted on a sample of 413 undergraduates who completed the SNAP twice, separated by one week. Participants were randomly assigned to one of four groups who completed (1) a modified P&P version of the SNAP (SNAP-PP) twice (n = 106), (2) the SNAP-PP first and the SNAP-CAT second (n = 105), (3) the SNAP-CAT first and the SNAP-PP second (n = 102), and (4) the SNAP-CAT twice (n = 100). Results indicated that the SNAP-CAT was 58% and 60% faster than the traditional P&P version, at Times 1 and 2, respectively, and mean item savings across scales were 36% and 37%, respectively. These savings came with minimal cost to reliability or validity, and the two test forms were largely equivalent. Descriptive statistics, rank-ordering of scores, internal factor structure, and convergent/discriminant validity were highly comparable across testing modes and methods of scoring, and very few differences between forms replicated across testing sessions. In addition, participants overwhelmingly preferred the computerized version to the P&P version. However, several specific problems were identified for the Self-harm and Propriety scales of the SNAP-CAT that appeared to be broadly related to IRT calibration difficulties. Reasons for these anomalous findings are discussed, and follow-up studies are suggested. Despite these specific problems, the SNAP-CAT appears to be a viable alternative to the traditional P&P SNAP. (PsycINFO Database Record (c) 2003 APA, all rights reserved).

Effect of extra time on GRE® Quantitative and Verbal Scores (Research Report 03-13) - Bridgeman, B.; Cline, F.; Hessinger, J. (2003)
{PDF file, 88 KB}

The effect of item selection method on the variability of CAT’s ability estimates when item parameters are contaminated with measurement errors - Li, Y. H.; Schafer, W. D. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 275 KB}

The effects of model misfit in computerized classification test - Jiao, H.; Lau, A. C. (2003)
{PDF file, 432 KB}

The effects of model specification error in item response theory-based computerized classification test using sequential probability ratio test - Jiao, H. (Dissertation Abstracts International Section A: Humanities & Social Sciences, 2003)
This study investigated the effects of model specification error on classification accuracy, error rates, and average test length in Item Response Theory (IRT) based computerized classification test (CCT) using sequential probability ratio test (SPRT) in making binary decisions from examinees' dichotomous responses. This study consisted of three sub-studies. In each sub-study, one of the three unidimensional dichotomous IRT models, the 1-parameter logistic (IPL), the 2-parameter logistic (2PL), and the 3-parameter logistic (3PL) model was set as the true model and the other two models were treated as the misfit models. Item pool composition, test length, and stratum depth were manipulated to simulate different test conditions. To ensure the validity of the study results, the true model based CCTs using the true and the recalibrated item parameters were compared first to study the effect of estimation error in item parameters in CCTs. Then, the true model and the misfit model based CCTs were compared to accomplish the research goal, The results indicated that estimation error in item parameters did not affect classification results based on CCTs using SPRT. The effect of model specification error depended on the true model, the misfit model, and the item pool composition. When the IPL or the 2PL IRT model was the true model, the use of another IRT model had little impact on the CCT results. When the 3PL IRT model was the true model, the use of the 1PL model raised the false positive error rates. The influence of using the 2PL instead of the 3PL model depended on the item pool composition. When the item discrimination parameters varied greatly from uniformity of one, the use of the 2PL IRT model raised the false negative error rates to above the nominal level. In the simulated test conditions with test length and item exposure constraints, using a misfit model in CCTs most often affected the average test length. Its effects on error rates and classification accuracy were negligible. It was concluded that in CCTs using SPRT, IRT model selection and evaluation is indispensable (PsycINFO Database Record (c) 2004 APA, all rights reserved).

Effects of test administration mode on item parameter estimates - Yi, Q.; Harris, D. J.; Wang, T.; Ban, J-C. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 233 KB}

Evaluating a new approach to detect aberrant responses in CAT - Lu, Y.,; Robin, F. (Paper presented at the annual meeting of the American Educational Research Association, 2003)
stimation of Ability Level by Using Only Observable Quantities in Adaptive Testing.

Evaluating computer-based test security by generalized item overlap rates - Zhang, J.; Lu, T. (Paper presented at the annual meeting of the American Educational Research Association, 2003)

Evaluating computerized adaptive testing design for the MCAT with realistic simulated data - Lu, Y.,; Pitoniak, M.; Rizavi, S.; Way, W. D.; Steffan, M. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 985 KB}

Evaluating stability of online item calibrations under varying conditions - Thomasson, G. L. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)

Evaluating the comparability of English- and French-speaking examinees on a science achievement test administered using two-stage testing - Puhan, G.; Gierl, M. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
PDF file, 568 K

The evaluation of exposure control procedures for an operational CAT. - French, B. F.; Thompson, T. T. (Paper presented at the Annual Meeting of the American Educational Research Association, 2003)

An examination of exposure control and content balancing restrictions on item selection in CATs using the partial credit model - Davis, L. L.; Pastor, D. A.; Dodd, B. G.; Chiang, C.; Fitzpatrick, S. J. (Journal of Applied Measurement, 2003)
The purpose of the present investigation was to systematically examine the effectiveness of the Sympson-Hetter technique and rotated content balancing relative to no exposure control and no content rotation conditions in a computerized adaptive testing system (CAT) based on the partial credit model. A series of simulated fixed and variable length CATs were run using two data sets generated to multiple content areas for three sizes of item pools. The 2 (exposure control) X 2 (content rotation) X 2 (test length) X 3 (item pool size) X 2 (data sets) yielded a total of 48 conditions. Results show that while both procedures can be used with no deleterious effect on measurement precision, the gains in exposure control, pool utilization, and item overlap appear quite modest. Difficulties involved with setting the exposure control parameters in small item pools make questionable the utility of the Sympson-Hetter technique with similar item pools.
1529-7713Journal Article
Keywords: *Computers; *Educational Measurement; *Models, Theoretical; Automation; Decision Making; Humans; Reproducibility of Results

Exposure control using adaptive multi-stage item bundles - Luecht, RM (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 116 KB}

Exposure control using adaptive multi-stage item bundles - Luecht, RM (annual meeting of the National Council on Measurement in Education, 2003)

The feasibility of applying item response theory to measures of migraine impact: a re-analysis of three clinical studies - Bjorner, J. B.; Kosinski, M.; Ware, J. E., Jr. (Quality of Life Research, 2003)
BACKGROUND: Item response theory (IRT) is a powerful framework for analyzing multiitem scales and is central to the implementation of computerized adaptive testing. OBJECTIVES: To explain the use of IRT to examine measurement properties and to apply IRT to a questionnaire for measuring migraine impact--the Migraine Specific Questionnaire (MSQ). METHODS: Data from three clinical studies that employed the MSQ-version 1 were analyzed by confirmatory factor analysis for categorical data and by IRT modeling. RESULTS: Confirmatory factor analyses showed very high correlations between the factors hypothesized by the original test constructions. Further, high item loadings on one common factor suggest that migraine impact may be adequately assessed by only one score. IRT analyses of the MSQ were feasible and provided several suggestions as to how to improve the items and in particular the response choices. Out of 15 items, 13 showed adequate fit to the IRT model. In general, IRT scores were strongly associated with the scores proposed by the original test developers and with the total item sum score. Analysis of response consistency showed that more than 90% of the patients answered consistently according to a unidimensional IRT model. For the remaining patients, scores on the dimension of emotional function were less strongly related to the overall IRT scores that mainly reflected role limitations. Such response patterns can be detected easily using response consistency indices. Analysis of test precision across score levels revealed that the MSQ was most precise at one standard deviation worse than the mean impact level for migraine patients that are not in treatment. Thus, gains in test precision can be achieved by developing items aimed at less severe levels of migraine impact. CONCLUSIONS: IRT proved useful for analyzing the MSQ. The approach warrants further testing in a more comprehensive item pool for headache impact that would enable computerized adaptive testing.
0962-9343Journal Article
Keywords: *Sickness Impact Profile; Adolescent; Adult; Aged; Comparative Study; Cost of Illness; Factor Analysis, Statistical; Feasibility Studies; Female; Human; Male; Middle Aged; Migraine/*psychology; Models, Psychological; Psychometrics/instrumentation/*methods; Quality of Life/*psychology; Questionnaires; Support, Non-U.S. Gov't

A feasibility study of on-the-fly item generation in adaptive testing - Bejar, I. I.; Lawless, R. R.,; Morley, M. E.,; Wagner, M. E.,; Bennett R. E.,; Revuelta, J. (Journal of Technology, Learning, and Assessment, 2003)
{PDF file, 427 KB}

Implementing an alternative to Sympson-Hetter item-exposure control in constrained adaptive testing - Veldkamp, B. P.; van der Linden, W. J. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)

Implementing content constraints in alpha-stratified adaptive testing using a shadow test approach - van der Linden, W. J.; Chang, Hua-Hua (Applied Psychological Measurement, 2003)

Implementing the a-stratified method with b blocking in computerized adaptive testing with the generalized partial credit model - Yi, Q.; Wang, T.; Wang, S (Paper presented at the annual meeting of the American Educational Research Association, 2003)
#YI03-01 {PDF file, 496 KB}

Incorporation Of Content Balancing Requirements In Stratification Designs For Computerized Adaptive Testing - Leung, Chi-Keung; Chang, Hua-Hua; Hau, Kit-Tai (Educational and Psychological Measurement, 2003)
In computerized adaptive testing, the multistage a-stratified design advocates a new philosophy on pool management and item selection in which, contradictory to common practice, less discriminating items are used first. The method is effective in reducing item-overlap rate and enhancing pool utilization. This stratification method has been extended in different ways to deal with the practical issues of content constraints and the positive correlation between item difficulty and discrimination. Nevertheless, these modified designs on their own do not automatically satisfy content requirements. In this study, three stratification designs were examined in conjunction with three well developed content balancing methods. The performance of each of these nine combinational methods was evaluated in terms of their item security, measurement efficiency, and pool utilization. Results showed substantial differences in item-overlap rate and pool utilization among different methods. An optimal combination of stratification design and content balancing method is recommended.

Incorporation of Content Balancing Requirements in Stratification Designs for Computerized Adaptive Testing - Leung, C-K..; Chang, Hua-Hua; Hau, K-T. (Educational and Psychological Measurement, 2003)
Studied three stratification designs for computerized adaptive testing in conjunction with three well-developed content balancing methods. Simulation study results show substantial differences in item overlap rate and pool utilization among different methods. Recommends an optimal combination of stratification design and content balancing method. (SLD)
Keywords: computerized adaptive testing

Increasing the homogeneity of CAT’s item-exposure rates by minimizing or maximizing varied target functions while assembling shadow tests - Li, Y. H.; Schafer, W. D. (Paper presented at the annual meeting of the American Educational Research Association, 2003)
(PDF file, 418 K

Information theoretic approaches to item selection - Weissman, A. (Paper presented at the 13th international meeting of the Psychometric Society, 2003)

Issues in maintaining scale consistency for the CAT-ASVAB - Nicewander, W. A. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)

Item banking to improve, shorten and computerized self-reported fatigue: an illustration of steps to create a core item bank from the FACIT-Fatigue Scale - Lai, J-S.; Crane, P. K.; Cella, D.; Chang, C-H.; Bode, R. K.; Heinemann, A. W. (Quality of Life Research, 2003)
Fatigue is a common symptom among cancer patients and the general population. Due to its subjective nature, fatigue has been difficult to effectively and efficiently assess. Modern computerized adaptive testing (CAT) can enable precise assessment of fatigue using a small number of items from a fatigue item bank. CAT enables brief assessment by selecting questions from an item bank that provide the maximum amount of information given a person's previous responses. This article illustrates steps to prepare such an item bank, using 13 items from the Functional Assessment of Chronic Illness Therapy Fatigue Subscale (FACIT-F) as the basis. Samples included 1022 cancer patients and 1010 people from the general population. An Item Response Theory (IRT)-based rating scale model, a polytomous extension of the Rasch dichotomous model was utilized. Nine items demonstrating acceptable psychometric properties were selected and positioned on the fatigue continuum. The fatigue levels measured by these nine items along with their response categories covered 66.8% of the general population and 82.6% of the cancer patients. Although the operational CAT algorithms to handle polytomously scored items are still in progress, we illustrated how CAT may work by using nine core items to measure level of fatigue. Using this illustration, a fatigue measure comparable to its full-length 13-item scale administration was obtained using four items. The resulting item bank can serve as a core to which will be added a psychometrically sound and operational item bank covering the entire fatigue continuum.
0962-9343Journal Article
Keywords: *Health Status Indicators; *Questionnaires; Adult; Fatigue/*diagnosis/etiology; Female; Humans; Male; Middle Aged; Neoplasms/complications; Psychometrics; Research Support, Non-U.S. Gov't; Research Support, U.S. Gov't, P.H.S.; Sickness Impact Profile

Item exposure constraints for testlets in the verbal reasoning section of the MCAT - Davis, L. L.; Dodd, B. G. (Applied Psychological Measurement, 2003)
The current study examined item exposure control procedures for testlet scored reading passages in the Verbal Reasoning section of the Medical College Admission Test with four computerized adaptive testing (CAT) systems using the partial credit model. The first system used a traditional CAT using maximum information item selection. The second used random item selection to provide a baseline for optimal exposure rates. The third used a variation of Lunz and Stahl's randomization procedure. The fourth used Luecht and Nungester's computerized adaptive sequential testing (CAST) system. A series of simulated fixed-length CATs was run to determine the optimal item length selection procedure. Results indicated that both the randomization procedure and CAST performed well in terms of exposure control and measurement precision, with the CAST system providing the best overall solution when all variables were taken into consideration. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Adaptive Testing; Computer Assisted Testing; Entrance Examinations; Item Response Theory; Random Sampling; Reasoning; Verbal Ability computerized adaptive testing

Item pool design for computerized adaptive tests - Reckase, M. D. (Paper presented at the Annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 135 KB}

Item selection in polytomous CAT - Veldkamp, B. P. (New developments in psychometrics, 2003)
Keywords: computerized adaptive testing

Item selection in polytomous CAT - Veldkamp, B. P. (2003)
#VE03207 {PDF file, 79 KB}

Maintaining scale in computer adaptive testing - Smith, R. L.; Rizavi, S.; Paez, R.; Damiano, M.; Herbert, E. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
{PDF file, 367 KB}

A method to determine targets for multi-stage adaptive tests - Armstrong, R. D.; Roussos, L. (2003)
{PDF file, 207 KB}

Methods for item set selection in adaptive testing - Lu, Y.,; Rizavi, S. (Paper presented at the annual meeting of the National Council on Measurement in Education, 2003)
PDF file, 443 K

Multidimensional computerized adaptive testing in recovering reading and mathematics abilities - Li, Y. H.; Schafer, W. D. (Paper presented at the annual meeting of the American Educational Research Association, 2003)
{PDF file, 592 KB}

A multidimensional IRT mechanism for better understanding adaptive test behavior - Jodoin, M. (2003)

Online calibration and scale stability of a CAT program - Guo, F.; Wang, G. (Paper presented at the annual meeting of the American Educational Research Association, 2003)
{PDF file, 274 KB}

An optimal design approach to criterion-referenced computerized testing - Wiberg, M. (Journal of Educational Measurement, 2003)

Optimal stratification of item pools in α-stratified computerized adaptive testing - Chang, Hua-Hua; van der Linden, W. J. (Applied Psychological Measurement, 2003)
A method based on 0-1 linear programming (LP) is presented to stratify an item pool optimally for use in α-stratified adaptive testing. Because the 0-1 LP model belongs to the subclass of models with a network flow structure, efficient solutions are possible. The method is applied to a previous item pool from the computerized adaptive testing (CAT) version of the Graduate Record Exams (GRE) Quantitative Test. The results indicate that the new method performs well in practical situations. It improves item exposure control, reduces the mean squared error in the θ estimates, and increases test reliability. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)
Keywords: Adaptive Testing; Computer Assisted Testing; Item Content (Test); Item Response Theory; Mathematical Modeling; Test Construction computerized adaptive testing