01395nas a2200181 4500008004500000022001400045245008700059210006900146490000700215520080200222653003401024653002501058653001901083653001901102653001601121100002101137856005501158 2024 Engldsh  a2165-659200aThe Influence of Computerized Adaptive Testing on Psychometric Theory and Practice0 aInfluence of Computerized Adaptive Testing on Psychometric Theor0 v113 a

The major premise of this article is that part of the stimulus for the evolution of psychometric theory since the 1950s was the introduction of the concept of computerized adaptive testing (CAT) or its earlier non-CAT variations. The conceptual underpinnings of CAT that had the most influence on psychometric theory was the shift of emphasis from the test (or test score) as the focus of analysis to the test item (or item score). The change in focus allowed a change in the way that test results are conceived of as measurements. It also resolved the conflict among a number of ideas that were present in the early work on psychometric theory. Some of the conflicting ideas are summarized below to show how work on the development of CAT resolved some of those conflicts.

 

10acomputerized adaptive testing10aItem Response Theory10aparadigm shift10ascaling theory10atest design1 aReckase, Mark, D uhttps://jcatpub.net/index.php/jcat/issue/view/34/900666nas a2200193 4500008004500000022001400045245007100059210006700130490000700197653002100204653002900225653002500254653003900279653001600318100001400334700002200348700002400370856007800394 2023 Engldsh  a2165-659200aAn Extended Taxonomy of Variants of Computerized Adaptive Testing0 aExtended Taxonomy of Variants of Computerized Adaptive Testing0 v1010aAdaptive Testing10aevidence-centered design10aItem Response Theory10aknowledge-based model construction10amissingness1 aLevy, Roy1 aBehrens, John, T.1 aMislevy, Robert, J. uhttp://iacat.org/extended-taxonomy-variants-computerized-adaptive-testing00480nas a2200133 4500008003900000245007300039210006700112490000600179653003400185653001300219653003500232100002600267856005300293 2022 d00aThe (non)Impact of Misfitting Items in Computerized Adaptive Testing0 anonImpact of Misfitting Items in Computerized Adaptive Testing0 v910acomputerized adaptive testing10aitem fit10athree-parameter logistic model1 aDeMars, Christine, E. uhttps://jcatpub.net/index.php/jcat/issue/view/2601609nas a2200205 4500008004500000022001400045245005000059210004900109300001000158490000600168520099300174653003501167653003401202653002301236653001901259653002701278100002301305700001501328856006001343 2019 Engldsh  a2165-659200aTime-Efficient Adaptive Measurement of Change0 aTimeEfficient Adaptive Measurement of Change a15-340 v73 a

The adaptive measurement of change (AMC) refers to the use of computerized adaptive testing (CAT) at multiple occasions to efficiently assess a respondent’s improvement, decline, or sameness from occasion to occasion. Whereas previous AMC research focused on administering the most informative item to a respondent at each stage of testing, the current research proposes the use of Fisher information per time unit as an item selection procedure for AMC. The latter procedure incorporates not only the amount of information provided by a given item but also the expected amount of time required to complete it. In a simulation study, the use of Fisher information per time unit item selection resulted in a lower false positive rate in the majority of conditions studied, and a higher true positive rate in all conditions studied, compared to item selection via Fisher information without accounting for the expected time taken. Future directions of research are suggested.

10aadaptive measurement of change10acomputerized adaptive testing10aFisher information10aitem selection10aresponse-time modeling1 aFinkelman, Matthew1 aWang, Chun uhttp://iacat.org/jcat/index.php/jcat/article/view/73/3505514nas a2200193 4500008004100000245009000041210006900131260005500200520480200255653002305057653001905080653002105099653002605120100002205146700002005168700001605188700001905204856009705223 2017 eng d00aAdaptive Item and Feedback Selection in Personalized Learning with a Network Approach0 aAdaptive Item and Feedback Selection in Personalized Learning wi aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Personalized learning is a term used to describe educational systems that adapt student-specific curriculum sequencing, pacing, and presentation based on their unique backgrounds, knowledge, preferences, interests, and learning goals. (Chen, 2008; Netcoh, 2016). The technological approach to personalized learning provides data-driven models to incorporate these adaptations automatically. Examples of applications include online learning systems, educational games, and revision-aid systems. In this study we introduce Bayesian networks as a methodology to implement an adaptive framework within a personalized learning environment. Existing ideas from Computerized Adaptive Testing (CAT) with Item Response Theory (IRT), where choices about content provision are based on maximizing information, are related to the goals of personalized learning environments. Personalized learning entails other goals besides efficient ability estimation by maximizing information, such as an adaptive configuration of preferences and feedback to the student. These considerations will be discussed and their application in networks will be illustrated.

Adaptivity in Personalized Learning.In standard CAT’s there is a focus on selecting items that provide maximum information about the ability of an individual at a certain point in time (Van der Linden & Glas, 2000). When learning is the main goal of testing, alternative adaptive item selection methods were explored by Eggen (2012). The adaptive choices made in personalized learning applications require additional adaptivity with respect to the following aspects; the moment of feedback, the kind of feedback, and the possibility for students to actively influence the learning process.

Bayesian Networks and Personalized Learning.Personalized learning aims at constructing a framework to incorporate all the aspects mentioned above. Therefore, the goal of this framework is not only to focus on retrieving ability estimates by choosing items on maximum information, but also to construct a framework that allows for these other factors to play a role. Plajner and Vomlel (2016) have already applied Bayesian Networks to adaptive testing, selecting items with help of entropy reduction. Almond et al. (2015) provide a reference work on Bayesian Networks in Educational Assessment. Both acknowledge the potential of the method in terms of features such as modularity options to build finer-grained models. IRT does not allow to model sub-skills very easily and to gather information on fine-grained level, due to its dependency on the assumption of generally one underlying trait. The local independence assumption in IRT implies being interested in mainly the student’s overall ability on the subject of interest. When the goal is to improve student’s learning, we are not just interested in efficiently coming to their test score on a global subject. One wants a model that is able to map educational problems and talents in detail over the whole educational program, while allowing for dependency between items. The moment in time can influence topics to be better mastered than others, and this is exactly what we can to get out of a model. The possibility to model flexible structures, estimate abilities on a very detailed level for sub-skills and to easily incorporate other variables such as feedback in Bayesian Networks makes it a very promising method for making adaptive choices in personalized learning. It is shown in this research how item and feedback selection can be performed with help of the promising Bayesian Networks. A student involvement possibility is also introduced and evaluated.

References

Almond, R. G., Mislevy, R. J., Steinberg, L. S., Yan, D., & Williamson, D. M. (2015). Bayesian Networks in Educational Assessment. Test. New York: Springer Science+Business Media. http://doi.org/10.1007/978-0-387-98138-3

Eggen, T.J.H.M. (2012) Computerized Adaptive Testing Item Selection in Computerized Adaptive Learning Systems. In: Eggen. TJHM & Veldkamp, BP.. (Eds). Psychometrics in Practice at RCEC. Enschede: RCEC

Netcoh, S. (2016, March). “What Do You Mean by ‘Personalized Learning?’. Croscutting Conversations in Education – Research, Reflections & Practice. Blogpost.

Plajner, M., & Vomlel, J. (2016). Student Skill Models in Adaptive Testing. In Proceedings of the Eighth International Conference on Probabilistic Graphical Models (pp. 403-414).

Van der Linden, W. J., & Glas, C. A. (2000). Computerized adaptive testing: Theory and practice. Dordrecht: Kluwer Academic Publishers.

Session Video

10afeedback selection10aitem selection10anetwork approach10apersonalized learning1 avan Buuren, Nikky1 aStraat, Hendrik1 aEggen, Theo1 aFox, Jean-Paul uhttp://iacat.org/adaptive-item-and-feedback-selection-personalized-learning-network-approach02709nas a2200157 4500008004100000245007100041210006900112260005500181520213800236653000802374653002002382653001402402100002002416700002602436856008902462 2017 eng d00aAnalysis of CAT Precision Depending on Parameters of the Item Pool0 aAnalysis of CAT Precision Depending on Parameters of the Item Po aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

The purpose of this research project is to analyze the measurement precision of a latent variable depending on parameters of the item pool. The influence of the following factors is analyzed:

Factor A – range of variation of items in the pool. This factor varies on three levels with the following ranges in logits: a1 – [-3.0; +3.0], a2 - [-4.0; +4.0], a3 - [-5.0; +5.0].

Factor B – number of items in the pool. The factor varies on six levels with the following number of items for every factor: b1 - 128, b2 - 256, b3 – 512, b4 - 1024, b5 – 2048, b6 – 4096. The items are evenly distributed in each of the variation ranges.

Factor C – examinees’ proficiency varies at 30 levels (c1, c2, …, c30), which are evenly distributed in the range [-3.0; +3.0] logit.

The investigation was based on a simulation experiment within the framework of the theory of latent variables.

Response Y is the precision of measurement of examinees’ proficiency, which is calculated as the difference between the true levels of examinees’ proficiency and estimates obtained by means of adaptive testing. Three factor ANOVA was used for data processing.

The following results were obtained:

1. Factor A is significant. Ceteris paribus, the greater the range of variation of items in the pool, the higher the estimation precision is.

2. Factor B is significant. Ceteris paribus, the greater the number of items in the pool, the higher the estimation precision is.

3. Factor C is statistically insignificant at level α = .05. It means that the precision of estimation of examinees’ proficiency is the same within the range of their variation.

4. The only significant interaction among all interactions is AB. The significance of this interaction is explained by the fact that increasing the number of items in the pool decreases the effect of the range of variation of items in the pool. 

Session Video

10aCAT10aItem parameters10aPrecision1 aMaslak, Anatoly1 aPozdniakov, Stanislav uhttps://drive.google.com/file/d/1Bwe58kOQRgCSbB8x6OdZTDK4OIm3LQI3/view?usp=drive_web02607nas a2200133 4500008004100000245004800041210004700089260005500136520214600191653002002337653002402357100002102381856007102402 2017 eng d00aDeveloping a CAT: An Integrated Perspective0 aDeveloping a CAT An Integrated Perspective aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Most resources on computerized adaptive testing (CAT) tend to focus on psychometric aspects such as mathematical formulae for item selection or ability estimation. However, development of a CAT assessment requires a holistic view of project management, financials, content development, product launch and branding, and more. This presentation will develop such a holistic view, which serves several purposes, including providing a framework for validity, estimating costs and ROI, and making better decisions regarding the psychometric aspects.

Thompson and Weiss (2011) presented a 5-step model for developing computerized adaptive tests (CATs). This model will be presented and discussed as the core of this holistic framework, then applied to real-life examples. While most CAT research focuses on developing new quantitative algorithms, this presentation is instead intended to help researchers evaluate and select algorithms that are most appropriate for their needs. It is therefore ideal for practitioners that are familiar with the basics of item response theory and CAT, and wish to explore how they might apply these methodologies to improve their assessments.

Steps include:

1. Feasibility, applicability, and planning studies

2. Develop item bank content or utilize existing bank

3. Pretest and calibrate item bank

4. Determine specifications for final CAT

5. Publish live CAT.

So, for example, Step 1 will contain simulation studies which estimate item bank requirements, which then can be used to determine costs of content development, which in turn can be integrated into an estimated project cost timeline. Such information is vital in determining if the CAT should even be developed in the first place.

References

Thompson, N. A., & Weiss, D. J. (2011). A Framework for the Development of Computerized Adaptive Tests. Practical Assessment, Research & Evaluation, 16(1). Retrieved from http://pareonline.net/getvn.asp?v=16&n=1.

Session Video

10aCAT Development10aintegrated approach1 aThompson, Nathan uhttps://drive.google.com/open?id=1Jv8bpH2zkw5TqSMi03e5JJJ98QtXf-Cv06631nas a2200157 4500008004100000245011400041210006900155260005500224520601700279653001106296653004106307653001906348100001706367700001806384856007106402 2017 eng d00aEfficiency of Item Selection in CD-CAT Based on Conjunctive Bayesian Network Modeling Hierarchical attributes0 aEfficiency of Item Selection in CDCAT Based on Conjunctive Bayes aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Cognitive diagnosis models (CDM) aim to diagnosis examinee’s mastery status of multiple fine-grained skills. As new development for cognitive diagnosis methods emerges, much attention is given to cognitive diagnostic computerized adaptive testing (CD-CAT) as well. The topics such as item selection methods, item exposure control strategies, and online calibration methods, which have been wellstudied for traditional item response theory (IRT) based CAT, are also investigated in the context of CD-CAT (e.g., Xu, Chang, & Douglas, 2003; Wang, Chang, & Huebner, 2011; Chen et al., 2012).

In CDM framework, some researchers suggest to model structural relationship between cognitive skills, or namely, attributes. Especially, attributes can be hierarchical, such that some attributes must be acquired before the subsequent ones are mastered. For example, in mathematics, addition must be mastered before multiplication, which gives a hierarchy model for addition skill and multiplication skill. Recently, new CDMs considering attribute hierarchies have been suggested including the Attribute Hierarchy Method (AHM; Leighton, Gierl, & Hunka, 2004) and the Hierarchical Diagnostic Classification Models (HDCM; Templin & Bradshaw, 2014).

Bayesian Networks (BN), the probabilistic graphical models representing the relationship of a set of random variables using a directed acyclic graph with conditional probability distributions, also provide an efficient framework for modeling the relationship between attributes (Culbertson, 2016). Among various BNs, conjunctive Bayesian network (CBN; Beerenwinkel, Eriksson, & Sturmfels, 2007) is a special kind of BN, which assumes partial ordering between occurrences of events and conjunctive constraints between them.

In this study, we propose using CBN for modeling attribute hierarchies and discuss the advantage of CBN for CDM. We then explore the impact of the CBN modeling on the efficiency of item selection methods for CD-CAT when the attributes are truly hierarchical. To this end, two simulation studies, one for fixed-length CAT and another for variable-length CAT, are conducted. For each studies, two attribute hierarchy structures with 5 and 8 attributes are assumed. Among the various item selection methods developed for CD-CAT, six algorithms are considered: posterior-weighted Kullback-Leibler index (PWKL; Cheng, 2009), the modified PWKL index (MPWKL; Kaplan, de la Torre, Barrada, 2015), Shannon entropy (SHE; Tatsuoka, 2002), mutual information (MI; Wang, 2013), posterior-weighted CDM discrimination index (PWCDI; Zheng & Chang, 2016) and posterior-weighted attribute-level CDM discrimination index (PWACDI; Zheng & Chang, 2016). The impact of Q-matrix structure, item quality, and test termination rules on the efficiency of item selection algorithms is also investigated. Evaluation measures include the attribute classification accuracy (fixed-length experiment) and test length of CDCAT until stopping (variable-length experiment).

The results of the study indicate that the efficiency of item selection is improved by directly modeling the attribute hierarchies using CBN. The test length until achieving diagnosis probability threshold was reduced to 50-70% for CBN based CAT compared to the CD-CAT assuming independence of attributes. The magnitude of improvement is greater when the cognitive model of the test includes more attributes and when the test length is shorter. We conclude by discussing how Q-matrix structure, item quality, and test termination rules affect the efficiency.

References

Beerenwinkel, N., Eriksson, N., & Sturmfels, B. (2007). Conjunctive bayesian networks. Bernoulli, 893- 909.

Chen, P., Xin, T., Wang, C., & Chang, H. H. (2012). Online calibration methods for the DINA model with independent attributes in CD-CAT. Psychometrika, 77(2), 201-222.

Cheng, Y. (2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74(4), 619-632.

Culbertson, M. J. (2016). Bayesian networks in educational assessment: the state of the field. Applied Psychological Measurement, 40(1), 3-21.

Kaplan, M., de la Torre, J., & Barrada, J. R. (2015). New item selection methods for cognitive diagnosis computerized adaptive testing. Applied Psychological Measurement, 39(3), 167-188.

Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The attribute hierarchy method for cognitive assessment: a variation on Tatsuoka's rule‐space approach. Journal of Educational Measurement, 41(3), 205-237.

Tatsuoka, C. (2002). Data analytic methods for latent partially ordered classification models. Journal of the Royal Statistical Society: Series C (Applied Statistics), 51(3), 337-350.

Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79(2), 317-339. Wang, C. (2013). Mutual information item selection method in cognitive diagnostic computerized adaptive testing with short test length. Educational and Psychological Measurement, 73(6), 1017-1035.

Wang, C., Chang, H. H., & Huebner, A. (2011). Restrictive stochastic item selection methods in cognitive diagnostic computerized adaptive testing. Journal of Educational Measurement, 48(3), 255-273.

Xu, X., Chang, H., & Douglas, J. (2003, April). A simulation study to compare CAT strategies for cognitive diagnosis. Paper presented at the annual meeting of National Council on Measurement in Education, Chicago.

Zheng, C., & Chang, H. H. (2016). High-efficiency response distribution–based item selection algorithms for short-length cognitive diagnostic computerized adaptive testing. Applied Psychological Measurement, 40(8), 608-624.

Session Video

10aCD-CAT10aConjuctive Bayesian Network Modeling10aitem selection1 aHan, Soo-Yun1 aYoo, Yun, Joo uhttps://drive.google.com/open?id=1RbO2gd4aULqsSgRi_VZudNN_edX82NeD01714nas a2200157 4500008004100000245007500041210006900116260005500185520116600240653000801406653002401414653001201438100002001450700001501470856007101485 2017 eng d00aFrom Blueprints to Systems: An Integrated Approach to Adaptive Testing0 aFrom Blueprints to Systems An Integrated Approach to Adaptive Te aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

For years, test blueprints have told test developers how many items and what types of items will be included in a test. Adaptive testing adopted this approach from paper testing, and it is reasonably useful. Unfortunately, 'how many items and what types of items' are not all the elements one should consider when choosing items for an adaptive test. To fill in gaps, practitioners have developed tools to allow an adaptive test to behave appropriately (i.e. examining exposure control, content balancing, item drift procedures, etc.). Each of these tools involves the use of a separate process external to the primary item selection process.

The use of these subsidiary processes makes item selection less optimal and makes it difficult to prioritize aspects of selection. This discussion describes systems-based adaptive testing. This approach uses metadata concerning items, test takers and test elements to select items. These elements are weighted by the stakeholders to shape an expanded blueprint designed for adaptive testing. 

Session Video

10aCAT10aintegrated approach10aKeynote1 aKingsbury, Gage1 aZara, Tony uhttps://drive.google.com/open?id=1CBaAfH4ES7XivmvrMjPeKyFCsFZOpQMJ03003nas a2200157 4500008004100000245007600041210006900117260005500186520243700241653002102678653002302699653002002722100001602742700001602758856007102774 2017 eng d00aGenerating Rationales to Support Formative Feedback in Adaptive Testing0 aGenerating Rationales to Support Formative Feedback in Adaptive  aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Computer adaptive testing offers many important benefits to support and promote life-long learning. Computers permit testing on-demand thereby allowing students to take the test at any time during instruction; items on computerized tests are scored immediately thereby providing students with instant feedback; computerized tests permit continuous administration thereby allowing students to have more choice about when they write their exams. But despite these important benefits, the advent of computer adaptive testing has also raised formidable challenges, particularly in the area of item development. Educators must have access to large numbers of diverse, high-quality test items to implement computerize adaptive testing because items are continuously administered to students. Hence, hundreds or even thousands of items are needed to develop the test item banks necessary for computer adaptive testing. Unfortunately, educational test items, as they are currently created, are time consuming and expensive to develop because each individual item is written, initially, by a content specialist and, then, reviewed, edited, and revised by groups of content specialists to ensure the items yield reliable and valid information. Hence, item development is one of the most important problems that must be solved before we can migrate to computer adaptive testing to support life-long learning because large numbers of high-quality, content-specific, test items are required.

One promising item development method that may be used to address this challenge is with automatic item generation. Automatic item generation is a relatively new but rapidly evolving research area where cognitive and psychometric modelling practices are used produce hundreds of new test items with the aid of computer technology. The purpose of our presentation is to describe a new methodology for generating both the items and the rationales required to solve each generated item in order to produce the feedback needed to support life-long learning. Our item generation methodology will first be described. To ensure our description is practical, the method will also be demonstrated using generated items from the health sciences to demonstrate how item generation can promote life-long learning for medical educators and practitioners.

 

Session Video

10aAdaptive Testing10aformative feedback10aItem generation1 aGierl, Mark1 aBulut, Okan uhttps://drive.google.com/open?id=1O5KDFtQlDLvhNoDr7X4JO4arpJkIHKUP01509nas a2200145 4500008004100000245003400041210003300075260005500108520107900163653002101242653001601263653001701279100002201296856004501318 2017 eng d00aGrow a Tiger out of Your CAT 0 aGrow a Tiger out of Your CAT aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

The main focus in the community of test developers and researchers is on improving adaptive test procedures and methodologies. Yet, the transition from research projects to larger-scale operational CATs is facing its own challenges. Usually, these operational CATs find their origin in government tenders. “Scalability”, “Interoperability” and “Transparency” are three keywords often found in these documents. Scalability is concerned with parallel system architectures which are based upon stateless selection algorithms. Design capacities often range from 10,000 to well over 100,000 concurrent students. Interoperability is implemented in standards like QTI, standards that were not designed with adaptive testing in mind. Transparency is being realized by open source software: the adaptive test should not be a black box. These three requirements often complicate the development of an adaptive test, or sometimes even conflict.

Session Video

10ainteroparability10aScalability10atransparency1 aVerschoor, Angela uhttp://iacat.org/grow-tiger-out-your-cat03610nas a2200169 4500008004100000245006900041210006600110260005500176520302300231653000803254653002403262653003303286100001503319700001803334700001703352856007103369 2017 eng d00aAn Imputation Approach to Handling Incomplete Computerized Tests0 aImputation Approach to Handling Incomplete Computerized Tests aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

As technology advances, computerized adaptive testing (CAT) is becoming increasingly popular as it allows tests to be tailored to an examinee’s ability.  Nevertheless, examinees might devise testing strategies to use CAT to their advantage.  For instance, if only the items that examinees answer count towards their score, then a higher theta score might be obtained by spending more time on items at the beginning of the test and skipping items at the end if time runs out. This type of gaming can be discouraged if examinees’ scores are lowered or “penalized” based on the amount of non-response.

The goal of this study was to devise a penalty function that would meet two criteria: 1) the greater the omit rate, the greater the penalty, and 2) examinees with the same ability and the same omit rate should receive the same penalty. To create the penalty, theta was calculated based on only the items the examinee responded to ( ).  Next, the expected number correct score (EXR) was obtained using  and the test characteristic curve. A penalized expected number correct score (E ) was obtained by multiplying EXR by the proportion of items the examinee responded to. Finally, the penalized theta ( ) was identified using the test characteristic curve. Based on   and the item parameters ( ) of an unanswered item, the likelihood of a correct response,  , is computed and employed to estimate the imputed score ( ) for the unanswered item.

Two datasets were used to generate tests with completion rates of 50%, 80%, and 90%.  The first dataset included real data where approximately 4,500 examinees responded to a 21 -item test which provided a baseline/truth. Sampling was done to achieve the three completion rate conditions. The second dataset consisted of simulated item scores for 50,000 simulees under a 1-2-4 multi-stage CAT design where each stage contained seven items. Imputed item scores for unanswered items were computed using a variety of values for G (and therefore T).  Three other approaches to handling unanswered items were also considered: all correct (i.e., T = 0), all incorrect (i.e., T = 1), and random scoring (i.e., T = 0.5).

The current study investigated the impact on theta estimates resulting from the proposed approach to handling unanswered items in a fixed-length CAT. In real testing situations, when examinees do not finish a test, it is hard to tell whether they tried diligently but ran out of time or whether they attempted to manipulate the scoring engine.  To handle unfinished tests with penalties, the proposed approach considers examinees’ abilities and incompletion rates. The results of this study provide direction for psychometric practitioners when considering penalties for omitted responses.

Session Video

10aCAT10aimputation approach10aincomplete computerized test1 aChen, Troy1 aHuang, Chi-Yu1 aLiu, Chunyan uhttps://drive.google.com/open?id=1vznZeO3nsZZK0k6_oyw5c9ZTP8uyGnXh03841nas a2200145 4500008004100000245014000041210006900181260005500250520324600305653000803551653001103559653002903570100002503599856007103624 2017 eng d00aIssues in Trait Range Coverage for Patient Reported Outcome Measure CATs - Extending the Ceiling for Above-average Physical Functioning0 aIssues in Trait Range Coverage for Patient Reported Outcome Meas aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

The use of a measure which fails to cover the upper range of functioning may produce results which can lead to serious misinterpretation. Scores produced by such a measure may fail to recognize significant improvement, or may not be able to demonstrate functioning commensurate with an important milestone. Accurate measurement of this range is critical for the assessment of physically active adults, e.g., athletes recovering from injury and active military personnel who wish to return to active service. Alternatively, a PF measure with a low ceiling might fail to differentiate patients in rehabilitation who continue to improve, but for whom their score ceilings due to the measurement used.

The assessment of physical function (PF) has greatly benefited from modern psychometric theory and resulting scales, such as the Patient-Reported Outcomes Measurement Information System (PROMIS®) PF instruments. While PROMIS PF has extended the range of function upwards relative to older “legacy” instruments, few PROMIS PF items asses high levels of function. We report here on the development of higher functioning items for the PROMIS PF bank.

An expert panel representing orthopedics, sports/military medicine, and rehabilitation reviewed existing instruments and wrote new items. After internal review, cognitive interviews were conducted with 24 individuals of average and high levels of physical function. The remaining candidate items were administered along with 50 existing PROMIS anchor items to an internet panel screened for low, average, and high levels of physical function (N = 1,600), as well as members of Boston-area gyms (N= 344). The resulting data was subjected to standard psychometric analysis, along with multiple linking methods to place the new items on the existing PF metric. The new items were added to the full PF bank for simulated computerized adaptive testing (CAT).

Item response data was collected on 54 candidate items. Items that exhibited local dependence (LD) or differential item functioning (DIF) related to gender, age, race, education, or PF status. These items were removed from consideration. Of the 50 existing PROMIS PF items, 31 were free of DIF and LD and used as anchors. The parameters for the remaining new candidate items were estimated twice: freelyestimated and linked with coefficients and fixed-anchor calibration. Both methods were comparable and had appropriate fit. The new items were added to the full PF bank for simulated CATs. The resulting CAT was able to extend the ceiling with high precision to a T-score of 68, suggesting accurate measurement for 97% of the general population.

Extending the range of items by which PF is measured will substantially improve measurement quality, applicability, and efficiency. The bank has incorporated these extension items and is available for use in research and clinics for brief CAT administration (see www.healthmeasures.net). Future research projects should focus on recovery trajectories of the measure for individuals with above average function who are recovering from injury.

Session Video

10aCAT10aIssues10aPatient Reported Outcome1 aGershon, Richard, C. uhttps://drive.google.com/open?id=1ZC02F-dIyYovEjzpeuRdoXDiXMLFRuKb01393nas a2200169 4500008004100000245003600041210003600077260005500113520088800168653000801056653002101064100002101085700001201106700001601118700001801134856007101152 2017 eng d00aItem Pool Design and Evaluation0 aItem Pool Design and Evaluation aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Early work on CAT tended to use existing sets of items which came from fixed length test forms. These sets of items were selected to meet much different requirements than are needed for a CAT; decision making or covering a content domain. However, there was also some early work that suggested having items equally distributed over the range of proficiency that was of interest or concentrated at a decision point. There was also some work that showed that there was bias in proficiency estimates when an item pool was too easy or too hard. These early findings eventually led to work on item pool design and, more recently, on item pool evaluation. This presentation gives a brief overview of these topics to give some context for the following presentations in this symposium.

Session Video

10aCAT10aItem Pool Design1 aReckase, Mark, D1 aHe, Wei1 aXu, Jing-Ru1 aZhou, Xuechun uhttps://drive.google.com/open?id=1ZAsqm1yNZlliqxEHcyyqQ_vOSu20xxZs01708nas a2200145 4500008004100000245006200041210006200103260005500165520122100220653000801441653001401449653003001463100002101493856004801514 2017 eng d00aItem Selection Strategies for Developing CAT in Indonesia0 aItem Selection Strategies for Developing CAT in Indonesia aNiigata JapanbNiiagata Seiryo Universityc08/20173 a

Recently, development of computerized testing in Indonesia is quiet promising for the future. Many government institutions used the technology for recruitment. Starting from Indonesian Army acknowledged the benefits of computerized adaptive testing (CAT) over conventional test administration, ones of the issues of selection the first item have taken place of attention. Due to CAT’s basic philosophy, several methods can be used to select the first item such as educational level, ability estimation from item simulation, or other methods. In this case, the question is remains how apply the methods most effective in the context of constrained adaptive testing. This paper reviews such strategies that appeared in the relevant literature. The focus of this paper is on studies that have been conducted in order to evaluate the effectiveness of item selection strategies for dichotomous scoring. In this paper, also discusses the strength and weaknesses of each strategy group using examples from simulation studies. No new research is presented but rather a compendium of models is reviewed in term of learning in the newcomer context, a wide view of first item selection strategies.

 

10aCAT10aIndonesia10aitem selection strategies1 aChandra, Istiani uhttps://www.youtube.com/watch?v=2KuFrRATq9Q00666nas a2200193 4500008003900000022001400039245008000053210006900133300001000202490000600212653004000218653002600258653003300284653002600317653002100343100002200364700002600386856006000412 2017 d a2165-659200aLatent-Class-Based Item Selection for Computerized Adaptive Progress Tests0 aLatentClassBased Item Selection for Computerized Adaptive Progre a22-430 v510acomputerized adaptive progress test10aitem selection method10aKullback-Leibler information10aLatent class analysis10alog-odds scoring1 avan Buuren, Nikky1 aEggen, Theo, J. H. M. uhttp://iacat.org/jcat/index.php/jcat/article/view/62/2902365nas a2200277 4500008004100000245007100041210006900112260005500181520152900236653000801765653001501773653002801788100001501816700002101831700001501852700001401867700001601881700001501897700001601912700001601928700001601944700001801960700001901978700001901997856007102016 2017 eng d00aNew Challenges (With Solutions) and Innovative Applications of CAT0 aNew Challenges With Solutions and Innovative Applications of CAT aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Over the past several decades, computerized adaptive testing (CAT) has profoundly changed the administration of large-scale aptitude tests, state-wide achievement tests, professional licensure exams, and health outcome measures. While many challenges of CAT have been successfully addressed due to the continual efforts of researchers in the field, there are still many remaining, longstanding challenges that have yet to be resolved. This symposium will begin with three presentations, each of which provides a sound solution to one of the unresolved challenges. They are (1) item calibration when responses are “missing not at random” from CAT administration; (2) online calibration of new items when person traits have non-ignorable measurement error; (3) establishing consistency and asymptotic normality of latent trait estimation when allowing item response revision in CAT. In addition, this symposium also features innovative applications of CAT. In particular, there is emerging interest in using cognitive diagnostic CAT to monitor and detect learning progress (4th presentation). Last but not least, the 5th presentation illustrates the power of multidimensional polytomous CAT that permits rapid identification of hospitalized patients’ rehabilitative care needs in health outcomes measurement. We believe this symposium covers a wide range of interesting and important topics in CAT.

Session Video

10aCAT10achallenges10ainnovative applications1 aWang, Chun1 aWeiss, David, J.1 aZhang, Xue1 aTao, Jian1 aHe, Yinhong1 aChen, Ping1 aWang, Shiyu1 aZhang, Susu1 aLin, Haiyan1 aGao, Xiaohong1 aChang, Hua-Hua1 aShang, Zhuoran uhttps://drive.google.com/open?id=1Wvgxw7in_QCq_F7kzID6zCZuVXWcFDPa03804nas a2200169 4500008004100000245008600041210006900127260005500196520318900251653000903440653000803449653002503457100002503482700001703507700001703524856009303541 2017 eng d00aNew Results on Bias in Estimates due to Discontinue Rules in Intelligence Testing0 aNew Results on Bias in Estimates due to Discontinue Rules in Int aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

The presentation provides new results on a form of adaptive testing that is used frequently in intelligence testing. In these tests, items are presented in order of increasing difficulty, and the presentation of items is adaptive in the sense that each subtest session is discontinued once a test taker produces a certain number of incorrect responses in sequence. The subsequent (not observed) responses are commonly scored as wrong for that subtest, even though the test taker has not seen these. Discontinuation rules allow a certain form of adaptiveness both in paper-based and computerbased testing, and help reducing testing time.

Two lines of research that are relevant are studies that directly assess the impact of discontinuation rules, and studies that more broadly look at the impact of scoring rules on test results with a large number of not administered or not reached items. He & Wolf (2012) compared different ability estimation methods for this type of discontinuation rule adaptation of test length in a simulation study. However, to our knowledge there has been no rigorous analytical study of the underlying distributional changes of the response variables under discontinuation rules. It is important to point out that the results obtained by He & Wolf (2012) agree with results presented by, for example, DeAyala, Plake & Impara (2001) as well as Rose, von Davier & Xu (2010) and Rose, von Davier & Nagengast (2016) in that ability estimates are biased most when scoring the not observed responses as wrong. Discontinuation rules combined with scoring the non-administered items as wrong is used operationally in several major intelligence tests, so more research is needed in order to improve this particular type of adaptiveness in the testing practice.

The presentation extends existing research on adaptiveness by discontinue-rules in intelligence tests in multiple ways: First, a rigorous analytical study of the distributional properties of discontinue-rule scored items is presented. Second, an extended simulation is presented that includes additional alternative scoring rules as well as bias-corrected ability estimators that may be suitable to improve results for discontinue-rule scored intelligence tests.

References: DeAyala, R. J., Plake, B. S., & Impara, J. C. (2001). The impact of omitted responses on the accuracy of ability estimation in item response theory. Journal of Educational Measurement, 38, 213-234.

He, W. & Wolfe, E. W. (2012). Treatment of Not-Administered Items on Individually Administered Intelligence Tests. Educational and Psychological Measurement, Vol 72, Issue 5, pp. 808 – 826. DOI: 10.1177/0013164412441937

Rose, N., von Davier, M., & Xu, X. (2010). Modeling non-ignorable missing data with item response theory (IRT; ETS RR-10-11). Princeton, NJ: Educational Testing Service.

Rose, N., von Davier, M., & Nagengast, B. (2016) Modeling omitted and not-reached items in irt models. Psychometrika. doi:10.1007/s11336-016-9544-7

Session Video

10aBias10aCAT10aIntelligence Testing1 avon Davier, Matthias1 aCho, Youngmi1 aPan, Tianshu uhttp://iacat.org/new-results-bias-estimates-due-discontinue-rules-intelligence-testing-004515nas a2200181 4500008004100000245010800041210006900149260005500218520381200273653000804085653002404093653002604117100001604143700001204159700001604171700002004187856012604207 2017 eng d00aUsing Computerized Adaptive Testing to Detect Students’ Misconceptions: Exploration of Item Selection0 aUsing Computerized Adaptive Testing to Detect Students Misconcep aNiigata, JapanbNiigata Seiryo Universityc08/20173 a

Owning misconceptions impedes learning, thus detecting misconceptions through assessments is crucial to facilitate teaching. However, most computerized adaptive testing (CAT) applications to diagnose examinees’ attribute profiles focus on whether examinees mastering correct concepts or not. In educational scenario, teachers and students have to figure out the misconceptions underlying incorrect answers after obtaining the scores from assessments and then correct the corresponding misconceptions. The Scaling Individuals and Classifying Misconceptions (SICM) models proposed by Bradshaw and Templin (2014) fill this gap. SICMs can identify a student’s misconceptions directly from the distractors of multiple-choice questions and report whether s/he own the misconceptions or not. Simultaneously, SICM models are able to estimate a continuous ability within the item response theory (IRT) framework to fulfill the needs of policy-driven assessment systems relying on scaling examinees’ ability. However, the advantage of providing estimations for two types of latent variables also causes complexity of model estimation. More items are required to achieve the same accuracies for both classification and estimation compared to dichotomous DCMs and to IRT, respectively. Thus, we aim to develop a CAT using the SICM models (SICM-CAT) to estimate students’ misconceptions and continuous abilities simultaneously using fewer items than a linear test.

To achieve this goal, in this study, our research questions mainly focus on establishing several item selection rules that target on providing both accurate classification results and continuous ability estimations using SICM-CAT. The first research question is which information criterion to be used. The Kullback–Leibler (KL) divergence is the first choice, as it can naturally combine the continuous and discrete latent variables. Based on this criterion, we propose an item selection index that can nicely integrate the two types of information. Based on this index, the items selected in real time could discriminate the examinee’s current misconception profile and ability estimates from other possible estimates to the most extent. The second research question is about how to adaptively balance the estimations of the misconception profile and the continuous latent ability. Mimic the idea of the Hybrid Design proposed by Wang et al. (2016), we propose a design framework which makes the item selection transition from the group-level to the item-level. We aim to explore several design questions, such as how to select the transiting point and which latent variable estimation should be targeted first.

Preliminary results indicated that the SICM-CAT based on the proposed item selection index could classify examinees into different latent classes and measure their latent abilities compared with the random selection method more accurately and reliably under all the simulation conditions. We plan to compare different CAT designs based on our proposed item selection rules with the best linear test as the next step. We expect that the SICM-CAT is able to use shorter test length while retaining the same accuracies and reliabilities.

References

Bradshaw, L., & Templin, J. (2014). Combining item response theory and diagnostic classification models: A psychometric model for scaling ability and diagnosing misconceptions. Psychometrika, 79(3), 403-425.

Wang, S., Lin, H., Chang, H. H., & Douglas, J. (2016). Hybrid computerized adaptive testing: from group sequential design to fully sequential design. Journal of Educational Measurement, 53(1), 45-62.

Session Video

10aCAT10aincorrect answering10aStudent Misconception1 aShen, Yawei1 aBao, Yu1 aWang, Shiyu1 aBradshaw, Laine uhttp://iacat.org/using-computerized-adaptive-testing-detect-students%E2%80%99-misconceptions-exploration-item-selection-000723nas a2200205 4500008004500000022001500045245013200060210006900192300000900261490000600270653002100276653003000297653003000327653001600357653002300373100002100396700002000417700002000437856006000457 2016 Engldsh  a2165-6592 00aEffect of Imprecise Parameter Estimation on Ability Estimation in a Multistage Test in an Automatic Item Generation Context 0 aEffect of Imprecise Parameter Estimation on Ability Estimation i a1-180 v410aAdaptive Testing10aautomatic item generation10aerrors in item parameters10aitem clones10amultistage testing1 aColvin, Kimberly1 aKeller, Lisa, A1 aRobin, Frederic uhttp://iacat.org/jcat/index.php/jcat/article/view/59/2700689nas a2200193 4500008004500000022001400045245012100059210006900180300001000249490000600259653003100265653002300296653002200319653003200341653002500373653001800398100001500416856006400431 2014 Engldsh  a2165-659200aDetecting Item Preknowledge in Computerized Adaptive Testing Using Information Theory and Combinatorial Optimization0 aDetecting Item Preknowledge in Computerized Adaptive Testing Usi a37-580 v210acombinatorial optimization10ahypothesis testing10aitem preknowledge10aKullback-Leibler divergence10asimulated annealing.10atest security1 aBelov, D I uhttp://www.iacat.org/jcat/index.php/jcat/article/view/36/1800623nas a2200169 4500008004100000245008900041210006900130260001200199653000800211653002500219653002700244653002100271653001200292100002300304700001600327856011000343 2011 eng d00aAdaptive Item Calibration and Norming: Unique Considerations of a Global Deployment0 aAdaptive Item Calibration and Norming Unique Considerations of a c10/201110aCAT10acommon item equating10aFigural Reasoning Test10aitem calibration10anorming1 aSchwall, Alexander1 aSinar, Evan uhttp://iacat.org/content/adaptive-item-calibration-and-norming-%0Bunique-considerations-global-deployment01161nas a2200157 4500008004100000245005700041210005600098520065200154653002100806653003400827653001500861653002500876100001300901700001500914856007400929 2011 eng d00acatR: An R Package for Computerized Adaptive Testing0 acatR An R Package for Computerized Adaptive Testing3 a

Computerized adaptive testing (CAT) is an active current research field in psychometrics and educational measurement. However, there is very little software available to handle such adaptive tasks. The R package catR was developed to perform adaptive testing with as much flexibility as possible, in an attempt to provide a developmental and testing platform to the interested user. Several item-selection rules and ability estimators are implemented. The item bank can be provided by the user or randomly generated from parent distributions of item parameters. Three stopping rules are available. The output can be graphically displayed.

10acomputer program10acomputerized adaptive testing10aEstimation10aItem Response Theory1 aMagis, D1 aRaîche, G uhttp://iacat.org/content/catr-r-package-computerized-adaptive-testing01629nas a2200145 4500008004100000245005200041210005000093260001200143520119300155653000801348653001601356653002001372100002301392856006801415 2011 eng d00aContinuous Testing (an avenue for CAT research)0 aContinuous Testing an avenue for CAT research c10/20113 a

Publishing an Adaptive Test

Problems with Publishing

Research Questions

10aCAT10aitem filter10aitem filtration1 aKingsbury, Gage, G uhttp://iacat.org/content/continuous-testing-avenue-cat-research02527nas a2200205 4500008004100000245011900041210006900160260001200229520179400241653000802035653000802043653003402051653003002085653000802115653003102123653001602154653001302170100002002183856011802203 2011 eng d00aFrom Reliability to Validity: Expanding Adaptive Testing Practice to Find the Most Valid Score for Each Test Taker0 aFrom Reliability to Validity Expanding Adaptive Testing Practice c10/20113 a

CAT is an exception to the traditional conception of validity. It is one of the few examples of individualized testing. Item difficulty is tailored to each examinee. The intent, however, is increased efficiency. Focus on reliability (reduced standard error); Equivalence with paper & pencil tests is valued; Validity is enhanced through improved reliability.

How Else Might We Individualize Testing Using CAT?

An ISV-Based View of Validity

Test Event -- An examinee encounters a series of items in a particular context.

CAT Goal: individualize testing to address CIV threats to score validity (i.e., maximize ISV).

Some Research Issues:

10aCAT10aCIV10aconstruct-irrelevant variance10aIndividual Score Validity10aISV10alow test taking motivation10aReliability10avalidity1 aWise, Steven, L uhttp://iacat.org/content/reliability-validity-expanding-adaptive-testing-practice-find-most-valid-score-each-test00770nas a2200145 4500008004100000245005700041210005700098260001200155520031700167653001500484100001700499700001300516700001900529856007600548 2011 eng d00aImpact of Item Drift on Candidate Ability Estimation0 aImpact of Item Drift on Candidate Ability Estimation c10/20113 a

For large operational pools, candidate ability estimates appear robust to item drift, especially under conditions that may represent ‘normal’ amounts of drift. Even with ‘extreme’ conditions of drift (e.g., 20% of items drifting 1.00 logits), decision consistency was still high.

10aitem drift1 aHagge, Sarah1 aWoo, Ada1 aDickison, Phil uhttp://iacat.org/content/impact-item-drift-candidate-ability-estimation01247nas a2200181 4500008004100000245012100041210006900162260001200231520055600243653003300799653000800832653001900840653003500859100001800894700001600912700002200928856011500950 2011 eng d00aItem Selection Methods based on Multiple Objective Approaches for Classification of Respondents into Multiple Levels0 aItem Selection Methods based on Multiple Objective Approaches fo c10/20113 a

Is it possible to develop new item selection methods which take advantage of the fact that we want to classify into multiple categories? New methods: Taking multiple points on the ability scale into account; Based on multiple objective approaches.

Conclusions

10aadaptive classification test10aCAT10aitem selection10asequential classification test1 aGroen, Maaike1 aEggen, Theo1 aVeldkamp, Bernard uhttp://iacat.org/content/item-selection-methods-based-multiple-objective-approaches-classification-respondents01099nas a2200169 4500008004100000245006600041210006600107260001200173520055300185653002600738653000800764653002100772653001700793653001000810100002200820856008700842 2011 eng d00aOptimal Calibration Designs for Computerized Adaptive Testing0 aOptimal Calibration Designs for Computerized Adaptive Testing c10/20113 a

Optimaztion

How can we exploit the advantages of Balanced Block Design while keeping the logistics manageable?

Homogeneous Designs: Overlap between test booklets as regular as possible

Conclusions:

10abalanced block design10aCAT10aitem calibration10aoptimization10aRasch1 aVerschoor, Angela uhttp://iacat.org/content/optimal-calibration-designs-computerized-adaptive-testing00684nas a2200217 4500008004100000245006000041210005800101260001200159653001700171653001700188653000800205653001500213653002500228653003200253653001700285100001800302700002100320700001600341700002400357856008500381 2011 eng d00aPractitioner’s Approach to Identify Item Drift in CAT0 aPractitioner s Approach to Identify Item Drift in CAT c10/201110aCUSUM method10aG2 statistic10aIPA10aitem drift10aitem parameter drift10aLord's chi-square statistic10aRaju's NCDIF1 aMeng, Huijuan1 aSteinkamp, Susan1 aJones, Paul1 aMatthews-Lopez, Joy uhttp://iacat.org/content/practitioner%E2%80%99s-approach-identify-item-drift-cat01507nas a2200217 4500008004100000245008100041210006900122300001200191490000700203520078900210653001100999653003401010653002201044653003501066653002101101653002101122100001901143700001401162700001601176856009701192 2010 eng d00aItem Selection and Hypothesis Testing for the Adaptive Measurement of Change0 aItem Selection and Hypothesis Testing for the Adaptive Measureme a238-2540 v343 a

Assessing individual change is an important topic in both psychological and educational measurement. An adaptive measurement of change (AMC) method had previously been shown to exhibit greater efficiency in detecting change than conventional nonadaptive methods. However, little work had been done to compare different procedures within the AMC framework. This study introduced a new item selection criterion and two new test statistics for detecting change with AMC that were specifically designed for the paradigm of hypothesis testing. In two simulation sets, the new methods for detecting significant change improved on existing procedures by demonstrating better adherence to Type I error rates and substantially better power for detecting relatively small change. 

10achange10acomputerized adaptive testing10aindividual change10aKullback–Leibler information10alikelihood ratio10ameasuring change1 aFinkelman, M D1 aWeiss, DJ1 aKim-Kang, G uhttp://iacat.org/content/item-selection-and-hypothesis-testing-adaptive-measurement-change-001520nas a2200181 4500008004100000020001400041245007400055210006900129300001200198490000800210520093200218653001401150653002401164653002301188100001501211700001901226856009301245 2009 eng d a0377-221700aA mixed integer programming model for multiple stage adaptive testing0 amixed integer programming model for multiple stage adaptive test a342-3500 v1933 aThe last decade has seen paper-and-pencil (P&P) tests being replaced by computerized adaptive tests (CATs) within many testing programs. A CAT may yield several advantages relative to a conventional P&P test. A CAT can determine the questions or test items to administer, allowing each test form to be tailored to a test taker's skill level. Subsequent items can be chosen to match the capability of the test taker. By adapting to a test taker's ability, a CAT can acquire more information about a test taker while administering fewer items. A Multiple Stage Adaptive test (MST) provides a means to implement a CAT that allows review before the administration. The MST format is a hybrid between the conventional P&P and CAT formats. This paper presents mixed integer programming models for MST assembly problems. Computational results with commercial optimization software will be given and advantages of the models evaluated.10aEducation10aInteger programming10aLinear programming1 aEdmonds, J1 aArmstrong, R D uhttp://iacat.org/content/mixed-integer-programming-model-multiple-stage-adaptive-testing03432nas a2200481 4500008004100000020004600041245013800087210006900225250001500294260000800309300001200317490000700329520191400336653002702250653002302277653003102300653001502331653001602346653001002362653002102372653002402393653002302417653003802440653001102478653002202489653001102511653001102522653000902533653003702542653002102579653003102600653002602631653001702657653003202674653001602706653002802722100001602750700001502766700001002781700001502791700002502806856011902831 2008 eng d a1532-821X (Electronic)0003-9993 (Linking)00aAssessing self-care and social function using a computer adaptive testing version of the pediatric evaluation of disability inventory0 aAssessing selfcare and social function using a computer adaptive a2008/04/01 cApr a622-6290 v893 aOBJECTIVE: To examine score agreement, validity, precision, and response burden of a prototype computer adaptive testing (CAT) version of the self-care and social function scales of the Pediatric Evaluation of Disability Inventory compared with the full-length version of these scales. DESIGN: Computer simulation analysis of cross-sectional and longitudinal retrospective data; cross-sectional prospective study. SETTING: Pediatric rehabilitation hospital, including inpatient acute rehabilitation, day school program, outpatient clinics; community-based day care, preschool, and children's homes. PARTICIPANTS: Children with disabilities (n=469) and 412 children with no disabilities (analytic sample); 38 children with disabilities and 35 children without disabilities (cross-validation sample). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Summary scores from prototype CAT applications of each scale using 15-, 10-, and 5-item stopping rules; scores from the full-length self-care and social function scales; time (in seconds) to complete assessments and respondent ratings of burden. RESULTS: Scores from both computer simulations and field administration of the prototype CATs were highly consistent with scores from full-length administration (r range, .94-.99). Using computer simulation of retrospective data, discriminant validity, and sensitivity to change of the CATs closely approximated that of the full-length scales, especially when the 15- and 10-item stopping rules were applied. In the cross-validation study the time to administer both CATs was 4 minutes, compared with over 16 minutes to complete the full-length scales. CONCLUSIONS: Self-care and social function score estimates from CAT administration are highly comparable with those obtained from full-length scale administration, with small losses in validity and precision and substantial decreases in administration time.10a*Disability Evaluation10a*Social Adjustment10aActivities of Daily Living10aAdolescent10aAge Factors10aChild10aChild, Preschool10aComputer Simulation10aCross-Over Studies10aDisabled Children/*rehabilitation10aFemale10aFollow-Up Studies10aHumans10aInfant10aMale10aOutcome Assessment (Health Care)10aReference Values10aReproducibility of Results10aRetrospective Studies10aRisk Factors10aSelf Care/*standards/trends10aSex Factors10aSickness Impact Profile1 aCoster, W J1 aHaley, S M1 aNi, P1 aDumas, H M1 aFragala-Pinkham, M A uhttp://iacat.org/content/assessing-self-care-and-social-function-using-computer-adaptive-testing-version-pediatric02707nas a2200217 4500008004100000020004100041245010800082210006900190250001500259300001100274490000600285520189600291653003802187653002902225653005702254653001102311653001302322653002402335100001302359856011702372 2008 eng d a1529-7713 (Print)1529-7713 (Linking)00aBinary items and beyond: a simulation of computer adaptive testing using the Rasch partial credit model0 aBinary items and beyond a simulation of computer adaptive testin a2008/01/09 a81-1040 v93 aPast research on Computer Adaptive Testing (CAT) has focused almost exclusively on the use of binary items and minimizing the number of items to be administrated. To address this situation, extensive computer simulations were performed using partial credit items with two, three, four, and five response categories. Other variables manipulated include the number of available items, the number of respondents used to calibrate the items, and various manipulations of respondents' true locations. Three item selection strategies were used, and the theoretically optimal Maximum Information method was compared to random item selection and Bayesian Maximum Falsification approaches. The Rasch partial credit model proved to be quite robust to various imperfections, and systematic distortions did occur mainly in the absence of sufficient numbers of items located near the trait or performance levels of interest. The findings further indicate that having small numbers of items is more problematic in practice than having small numbers of respondents to calibrate these items. Most importantly, increasing the number of response categories consistently improved CAT's efficiency as well as the general quality of the results. In fact, increasing the number of response categories proved to have a greater positive impact than did the choice of item selection method, as the Maximum Information approach performed only slightly better than the Maximum Falsification approach. Accordingly, issues related to the efficiency of item selection methods are far less important than is commonly suggested in the literature. However, being based on computer simulations only, the preceding presumes that actual respondents behave according to the Rasch model. CAT research could thus benefit from empirical studies aimed at determining whether, and if so, how, selection strategies impact performance.10a*Data Interpretation, Statistical10a*User-Computer Interface10aEducational Measurement/*statistics & numerical data10aHumans10aIllinois10aModels, Statistical1 aLange, R uhttp://iacat.org/content/binary-items-and-beyond-simulation-computer-adaptive-testing-using-rasch-partial-credit03309nas a2200433 4500008004100000020004600041245007700087210006900164250001500233260001100248300001200259490000700271520203200278653002702310653003002337653002102367653001002388653000902398653001502407653003602422653002102458653004402479653002402523653001102547653001102558653001302569653000902582653001602591653003002607653003002637653003102667100001502698700001302713700001502726700001402741700001502755700001402770856009102784 2008 eng d a1528-1159 (Electronic)0362-2436 (Linking)00aComputerized adaptive testing in back pain: Validation of the CAT-5D-QOL0 aComputerized adaptive testing in back pain Validation of the CAT a2008/05/23 cMay 20 a1384-900 v333 aSTUDY DESIGN: We have conducted an outcome instrument validation study. OBJECTIVE: Our objective was to develop a computerized adaptive test (CAT) to measure 5 domains of health-related quality of life (HRQL) and assess its feasibility, reliability, validity, and efficiency. SUMMARY OF BACKGROUND DATA: Kopec and colleagues have recently developed item response theory based item banks for 5 domains of HRQL relevant to back pain and suitable for CAT applications. The domains are Daily Activities (DAILY), Walking (WALK), Handling Objects (HAND), Pain or Discomfort (PAIN), and Feelings (FEEL). METHODS: An adaptive algorithm was implemented in a web-based questionnaire administration system. The questionnaire included CAT-5D-QOL (5 scales), Modified Oswestry Disability Index (MODI), Roland-Morris Disability Questionnaire (RMDQ), SF-36 Health Survey, and standard clinical and demographic information. Participants were outpatients treated for mechanical back pain at a referral center in Vancouver, Canada. RESULTS: A total of 215 patients completed the questionnaire and 84 completed a retest. On average, patients answered 5.2 items per CAT-5D-QOL scale. Reliability ranged from 0.83 (FEEL) to 0.92 (PAIN) and was 0.92 for the MODI, RMDQ, and Physical Component Summary (PCS-36). The ceiling effect was 0.5% for PAIN compared with 2% for MODI and 5% for RMQ. The CAT-5D-QOL scales correlated as anticipated with other measures of HRQL and discriminated well according to the level of satisfaction with current symptoms, duration of the last episode, sciatica, and disability compensation. The average relative discrimination index was 0.87 for PAIN, 0.67 for DAILY and 0.62 for WALK, compared with 0.89 for MODI, 0.80 for RMDQ, and 0.59 for PCS-36. CONCLUSION: The CAT-5D-QOL is feasible, reliable, valid, and efficient in patients with back pain. This methodology can be recommended for use in back pain research and should improve outcome assessment, facilitate comparisons across studies, and reduce patient burden.10a*Disability Evaluation10a*Health Status Indicators10a*Quality of Life10aAdult10aAged10aAlgorithms10aBack Pain/*diagnosis/psychology10aBritish Columbia10aDiagnosis, Computer-Assisted/*standards10aFeasibility Studies10aFemale10aHumans10aInternet10aMale10aMiddle Aged10aPredictive Value of Tests10aQuestionnaires/*standards10aReproducibility of Results1 aKopec, J A1 aBadii, M1 aMcKenna, M1 aLima, V D1 aSayre, E C1 aDvorak, M uhttp://iacat.org/content/computerized-adaptive-testing-back-pain-validation-cat-5d-qol01872nas a2200205 4500008003900000245005600039210005600095300001000151490000800161520124900169653002101418653003001439653002501469653001801494653002501512100001301537700001701550700002101567856007801588 2008 d00aComputerized Adaptive Testing of Personality Traits0 aComputerized Adaptive Testing of Personality Traits a12-210 v2163 a

A computerized adaptive testing (CAT) procedure was simulated with ordinal polytomous personality data collected using a
conventional paper-and-pencil testing format. An adapted Dutch version of the dominance scale of Gough and Heilbrun’s Adjective
Check List (ACL) was used. This version contained Likert response scales with five categories. Item parameters were estimated using Samejima’s graded response model from the responses of 1,925 subjects. The CAT procedure was simulated using the responses of 1,517 other subjects. The value of the required standard error in the stopping rule of the CAT was manipulated. The relationship between CAT latent trait estimates and estimates based on all dominance items was studied. Additionally, the pattern of relationships between the CAT latent trait estimates and the other ACL scales was compared to that between latent trait estimates based on the entire item pool and the other ACL scales. The CAT procedure resulted in latent trait estimates qualitatively equivalent to latent trait estimates based on all items, while a substantial reduction of the number of used items could be realized (at the stopping rule of 0.4 about 33% of the 36 items was used).

10aAdaptive Testing10acmoputer-assisted testing10aItem Response Theory10aLikert scales10aPersonality Measures1 aHol, A M1 aVorst, H C M1 aMellenbergh, G J uhttp://iacat.org/content/computerized-adaptive-testing-personality-traits01724nas a2200181 4500008004100000245011200041210006900153300001100222490000700233520107200240653003401312653001701346653001901363100001601382700001801398700001501416856011101431 2008 eng d00aThe D-optimality item selection criterion in the early stage of CAT: A study with the graded response model0 aDoptimality item selection criterion in the early stage of CAT A a88-1100 v333 aDuring the early stage of computerized adaptive testing (CAT), item selection criteria based on Fisher’s information often produce less stable latent trait estimates than the Kullback-Leibler global information criterion. Robustness against early stage instability has been reported for the D-optimality criterion in a polytomous CAT with the Nominal Response Model and is shown herein to be reproducible for the Graded Response Model. For comparative purposes, the A-optimality and the global information criteria are also applied. Their item selection is investigated as a function of test progression and item bank composition. The results indicate how the selection of specific item parameters underlies the criteria performances evaluated via accuracy and precision of estimation. In addition, the criteria item exposure rates are compared, without the use of any exposure controlling measure. On the account of stability, precision, accuracy, numerical simplicity, and less evidently, item exposure rate, the D-optimality criterion can be recommended for CAT.10acomputerized adaptive testing10aD optimality10aitem selection1 aPassos, V L1 aBerger, M P F1 aTan, F E S uhttp://iacat.org/content/d-optimality-item-selection-criterion-early-stage-cat-study-graded-response-model03424nas a2200385 4500008004100000020004100041245010600082210006900188250001500257260001200272300001000284490000700294520220300301653002702504653001502531653001002546653002102556653002402577653002802601653003802629653001102667653001102678653001102689653003902700653000902739653002402748653003102772653004002803100001802843700001502861700001302876700001702889700001402906856011802920 2008 eng d a0271-6798 (Print)0271-6798 (Linking)00aMeasuring physical functioning in children with spinal impairments with computerized adaptive testing0 aMeasuring physical functioning in children with spinal impairmen a2008/03/26 cApr-May a330-50 v283 aBACKGROUND: The purpose of this study was to assess the utility of measuring current physical functioning status of children with scoliosis and kyphosis by applying computerized adaptive testing (CAT) methods. Computerized adaptive testing uses a computer interface to administer the most optimal items based on previous responses, reducing the number of items needed to obtain a scoring estimate. METHODS: This was a prospective study of 77 subjects (0.6-19.8 years) who were seen by a spine surgeon during a routine clinic visit for progress spine deformity. Using a multidimensional version of the Pediatric Evaluation of Disability Inventory CAT program (PEDI-MCAT), we evaluated content range, accuracy and efficiency, known-group validity, concurrent validity with the Pediatric Outcomes Data Collection Instrument, and test-retest reliability in a subsample (n = 16) within a 2-week interval. RESULTS: We found the PEDI-MCAT to have sufficient item coverage in both self-care and mobility content for this sample, although most patients tended to score at the higher ends of both scales. Both the accuracy of PEDI-MCAT scores as compared with a fixed format of the PEDI (r = 0.98 for both mobility and self-care) and test-retest reliability were very high [self-care: intraclass correlation (3,1) = 0.98, mobility: intraclass correlation (3,1) = 0.99]. The PEDI-MCAT took an average of 2.9 minutes for the parents to complete. The PEDI-MCAT detected expected differences between patient groups, and scores on the PEDI-MCAT correlated in expected directions with scores from the Pediatric Outcomes Data Collection Instrument domains. CONCLUSIONS: Use of the PEDI-MCAT to assess the physical functioning status, as perceived by parents of children with complex spinal impairments, seems to be feasible and achieves accurate and efficient estimates of self-care and mobility function. Additional item development will be needed at the higher functioning end of the scale to avoid ceiling effects for older children. LEVEL OF EVIDENCE: This is a level II prospective study designed to establish the utility of computer adaptive testing as an evaluation method in a busy pediatric spine practice.10a*Disability Evaluation10aAdolescent10aChild10aChild, Preschool10aComputer Simulation10aCross-Sectional Studies10aDisabled Children/*rehabilitation10aFemale10aHumans10aInfant10aKyphosis/*diagnosis/rehabilitation10aMale10aProspective Studies10aReproducibility of Results10aScoliosis/*diagnosis/rehabilitation1 aMulcahey, M J1 aHaley, S M1 aDuffy, T1 aPengsheng, N1 aBetz, R R uhttp://iacat.org/content/measuring-physical-functioning-children-spinal-impairments-computerized-adaptive-testing01944nas a2200301 4500008004500000020001400045245012900059210006900188300001200257490000700269520093500276653002501211653002101236653002501257653003001282653003001312653001001342653001501352653002601367653002501393653002401418653001501442653001501457100001301472700001701485700002101502856011901523 2007 Engldsh  a0146-621600aComputerized adaptive testing for polytomous motivation items: Administration mode effects and a comparison with short forms0 aComputerized adaptive testing for polytomous motivation items Ad a412-4290 v313 aIn a randomized experiment (n=515), a computerized and a computerized adaptive test (CAT) are compared. The item pool consists of 24 polytomous motivation items. Although items are carefully selected, calibration data show that Samejima's graded response model did not fit the data optimally. A simulation study is done to assess possible consequences of model misfit. CAT efficiency was studied by a systematic comparison of the CAT with two types of conventional fixed length short forms, which are created to be good CAT competitors. Results showed no essential administration mode effects. Efficiency analyses show that CAT outperformed the short forms in almost all aspects when results are aggregated along the latent trait scale. The real and the simulated data results are very similar, which indicate that the real data results are not affected by model misfit. (PsycINFO Database Record (c) 2007 APA ) (journal abstract)10a2220 Tests & Testing10aAdaptive Testing10aAttitude Measurement10acomputer adaptive testing10aComputer Assisted Testing10aitems10aMotivation10apolytomous motivation10aStatistical Validity10aTest Administration10aTest Forms10aTest Items1 aHol, A M1 aVorst, H C M1 aMellenbergh, G J uhttp://iacat.org/content/computerized-adaptive-testing-polytomous-motivation-items-administration-mode-effects-and02871nas a2200313 4500008004100000020002200041245010100063210006900164250001500233260000800248300001200256490000700268520179500275653005102070653002002121653003702141653002602178653001902204653001102223653003002234653004602264653003502310653002802345653001302373100002102386700001702407700001502424856011802439 2007 eng d a0315-162X (Print)00aImproving patient reported outcomes using item response theory and computerized adaptive testing0 aImproving patient reported outcomes using item response theory a a2007/06/07 cJun a1426-310 v343 aOBJECTIVE: Patient reported outcomes (PRO) are considered central outcome measures for both clinical trials and observational studies in rheumatology. More sophisticated statistical models, including item response theory (IRT) and computerized adaptive testing (CAT), will enable critical evaluation and reconstruction of currently utilized PRO instruments to improve measurement precision while reducing item burden on the individual patient. METHODS: We developed a domain hierarchy encompassing the latent trait of physical function/disability from the more general to most specific. Items collected from 165 English-language instruments were evaluated by a structured process including trained raters, modified Delphi expert consensus, and then patient evaluation. Each item in the refined data bank will undergo extensive analysis using IRT to evaluate response functions and measurement precision. CAT will allow for real-time questionnaires of potentially smaller numbers of questions tailored directly to each individual's level of physical function. RESULTS: Physical function/disability domain comprises 4 subdomains: upper extremity, trunk, lower extremity, and complex activities. Expert and patient review led to consensus favoring use of present-tense "capability" questions using a 4- or 5-item Likert response construct over past-tense "performance"items. Floor and ceiling effects, attribution of disability, and standardization of response categories were also addressed. CONCLUSION: By applying statistical techniques of IRT through use of CAT, existing PRO instruments may be improved to reduce questionnaire burden on the individual patients while increasing measurement precision that may ultimately lead to reduced sample size requirements for costly clinical trials.10a*Rheumatic Diseases/physiopathology/psychology10aClinical Trials10aData Interpretation, Statistical10aDisability Evaluation10aHealth Surveys10aHumans10aInternational Cooperation10aOutcome Assessment (Health Care)/*methods10aPatient Participation/*methods10aResearch Design/*trends10aSoftware1 aChakravarty, E F1 aBjorner, J B1 aFries, J F uhttp://iacat.org/content/improving-patient-reported-outcomes-using-item-response-theory-and-computerized-adaptive02408nas a2200361 4500008004500000020001400045245011100059210006900170300001200239490000700251520135900258653001601617653002001633653001301653653002401666653002501690653001101715653001401726653001301740653001801753653002701771653001001798653001101808100001501819700001201834700001601846700001201862700001601874700001301890700001301903700001401916856011601930 2007 Engldsh  a1057-924900aThe initial development of an item bank to assess and screen for psychological distress in cancer patients0 ainitial development of an item bank to assess and screen for psy a724-7320 v163 aPsychological distress is a common problem among cancer patients. Despite the large number of instruments that have been developed to assess distress, their utility remains disappointing. This study aimed to use Rasch models to develop an item-bank which would provide the basis for better means of assessing psychological distress in cancer patients. An item bank was developed from eight psychological distress questionnaires using Rasch analysis to link common items. Items from the questionnaires were added iteratively with common items as anchor points and misfitting items (infit mean square > 1.3) removed, and unidimensionality assessed. A total of 4914 patients completed the questionnaires providing an initial pool of 83 items. Twenty items were removed resulting in a final pool of 63 items. Good fit was demonstrated and no additional factor structure was evident from the residuals. However, there was little overlap between item locations and person measures, since items mainly targeted higher levels of distress. The Rasch analysis allowed items to be pooled and generated a unidimensional instrument for measuring psychological distress in cancer patients. Additional items are required to more accurately assess patients across the whole continuum of psychological distress. (PsycINFO Database Record (c) 2007 APA ) (journal abstract)10a3293 Cancer10acancer patients10aDistress10ainitial development10aItem Response Theory10aModels10aNeoplasms10aPatients10aPsychological10apsychological distress10aRasch10aStress1 aSmith, A B1 aRush, R1 aVelikova, G1 aWall, L1 aWright, E P1 aStark, D1 aSelby, P1 aSharpe, M uhttp://iacat.org/content/initial-development-item-bank-assess-and-screen-psychological-distress-cancer-patients02208nas a2200229 4500008004100000020004600041245008500087210006900172260004500241300001000286490000600296520140300302653003401705653002301739653002601762653001701788653002601805100001701831700001201848700001501860856010301875 2007 eng d a1614-1881 (Print); 1614-2241 (Electronic)00aMethods for restricting maximum exposure rate in computerized adaptative testing0 aMethods for restricting maximum exposure rate in computerized ad bHogrefe & Huber Publishers GmbH: Germany a14-230 v33 aThe Sympson-Hetter (1985) method provides a means of controlling maximum exposure rate of items in Computerized Adaptive Testing. Through a series of simulations, control parameters are set that mark the probability of administration of an item on being selected. This method presents two main problems: it requires a long computation time for calculating the parameters and the maximum exposure rate is slightly above the fixed limit. Van der Linden (2003) presented two alternatives which appear to solve both of the problems. The impact of these methods in the measurement accuracy has not been tested yet. We show how these methods over-restrict the exposure of some highly discriminating items and, thus, the accuracy is decreased. It also shown that, when the desired maximum exposure rate is near the minimum possible value, these methods offer an empirical maximum exposure rate clearly above the goal. A new method, based on the initial estimation of the probability of administration and the probability of selection of the items with the restricted method (Revuelta & Ponsoda, 1998), is presented in this paper. It can be used with the Sympson-Hetter method and with the two van der Linden's methods. This option, when used with Sympson-Hetter, speeds the convergence of the control parameters without decreasing the accuracy. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive testing10aitem bank security10aitem exposure control10aoverlap rate10aSympson-Hetter method1 aBarrada, J R1 aOlea, J1 aPonsoda, V uhttp://iacat.org/content/methods-restricting-maximum-exposure-rate-computerized-adaptative-testing02408nas a2200289 4500008004100000020002200041245010800063210006900171260004500240300001000285490000700295520141000302653003201712653002501744653002501769653002501794653002601819653001801845653003701863653002101900653001301921100001501934700001401949700001901963700002001982856011602002 2007 eng d a1015-5759 (Print)00aPsychometric properties of an emotional adjustment measure: An application of the graded response model0 aPsychometric properties of an emotional adjustment measure An ap bHogrefe & Huber Publishers GmbH: Germany a39-460 v233 aItem response theory (IRT) provides valuable methods for the analysis of the psychometric properties of a psychological measure. However, IRT has been mainly used for assessing achievements and ability rather than personality factors. This paper presents an application of the IRT to a personality measure. Thus, the psychometric properties of a new emotional adjustment measure that consists of a 28-six graded response items is shown. Classical test theory (CTT) analyses as well as IRT analyses are carried out. Samejima's (1969) graded-response model has been used for estimating item parameters. Results show that the bank of items fulfills model assumptions and fits the data reasonably well, demonstrating the suitability of the IRT models for the description and use of data originating from personality measures. In this sense, the model fulfills the expectations that IRT has undoubted advantages: (1) The invariance of the estimated parameters, (2) the treatment given to the standard error of measurement, and (3) the possibilities offered for the construction of computerized adaptive tests (CAT). The bank of items shows good reliability. It also shows convergent validity compared to the Eysenck Personality Inventory (EPQ-A; Eysenck & Eysenck, 1975) and the Big Five Questionnaire (BFQ; Caprara, Barbaranelli, & Borgogni, 1993). (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive tests10aEmotional Adjustment10aItem Response Theory10aPersonality Measures10apersonnel recruitment10aPsychometrics10aSamejima's graded response model10atest reliability10avalidity1 aRubio, V J1 aAguado, D1 aHontangas, P M1 aHernández, J M uhttp://iacat.org/content/psychometric-properties-emotional-adjustment-measure-application-graded-response-model01959nas a2200265 4500008004100000020002200041245008200063210006900145260002600214300001000240490000700250520112900257653001501386653003401401653001401435653001701449653002401466653001501490653002201505653001501527100002301542700001301565700001801578856009701596 2006 eng d a1076-9986 (Print)00aAssembling a computerized adaptive testing item pool as a set of linear tests0 aAssembling a computerized adaptive testing item pool as a set of bSage Publications: US a81-990 v313 aTest-item writing efforts typically results in item pools with an undesirable correlational structure between the content attributes of the items and their statistical information. If such pools are used in computerized adaptive testing (CAT), the algorithm may be forced to select items with less than optimal information, that violate the content constraints, and/or have unfavorable exposure rates. Although at first sight somewhat counterintuitive, it is shown that if the CAT pool is assembled as a set of linear test forms, undesirable correlations can be broken down effectively. It is proposed to assemble such pools using a mixed integer programming model with constraints that guarantee that each test meets all content specifications and an objective function that requires them to have maximal information at a well-chosen set of ability values. An empirical example with a previous master pool from the Law School Admission Test (LSAT) yielded a CAT with nearly uniform bias and mean-squared error functions for the ability estimator and item-exposure rates that satisfied the target for all items in the pool. 10aAlgorithms10acomputerized adaptive testing10aitem pool10alinear tests10amathematical models10astatistics10aTest Construction10aTest Items1 avan der Linden, WJ1 aAriel, A1 aVeldkamp, B P uhttp://iacat.org/content/assembling-computerized-adaptive-testing-item-pool-set-linear-tests01943nas a2200241 4500008004100000020002200041245011200063210006900175260004100244300001200285490000700297520102300304653003401327653002401361653004701385653002401432653002101456653007001477100001301547700001401560700001001574856011701584 2006 eng d a0022-0655 (Print)00aComparing methods of assessing differential item functioning in a computerized adaptive testing environment0 aComparing methods of assessing differential item functioning in  bBlackwell Publishing: United Kingdom a245-2640 v433 aMantel-Haenszel and SIBTEST, which have known difficulty in detecting non-unidirectional differential item functioning (DIF), have been adapted with some success for computerized adaptive testing (CAT). This study adapts logistic regression (LR) and the item-response-theory-likelihood-ratio test (IRT-LRT), capable of detecting both unidirectional and non-unidirectional DIF, to the CAT environment in which pretest items are assumed to be seeded in CATs but not used for trait estimation. The proposed adaptation methods were evaluated with simulated data under different sample size ratios and impact conditions in terms of Type I error, power, and specificity in identifying the form of DIF. The adapted LR and IRT-LRT procedures are more powerful than the CAT version of SIBTEST for non-unidirectional DIF detection. The good Type I error control provided by IRT-LRT under extremely unequal sample sizes and large impact is encouraging. Implications of these and other findings are discussed. all rights reserved)10acomputerized adaptive testing10aeducational testing10aitem response theory likelihood ratio test10alogistic regression10atrait estimation10aunidirectional & non-unidirectional differential item functioning1 aLei, P-W1 aChen, S-Y1 aYu, L uhttp://iacat.org/content/comparing-methods-assessing-differential-item-functioning-computerized-adaptive-testing02399nas a2200241 4500008004100000020002200041245008500063210006900148260002500217300001200242490000700254520161500261653000801876653003401884653002601918653003001944653002601974100001402000700001402014700001602028700001502044856009802059 2006 eng d a0439-755X (Print)00aThe comparison among item selection strategies of CAT with multiple-choice items0 acomparison among item selection strategies of CAT with multiplec bScience Press: China a778-7830 v383 aThe initial purpose of comparing item selection strategies for CAT was to increase the efficiency of tests. As studies continued, however, it was found that increasing the efficiency of item bank using was also an important goal of comparing item selection strategies. These two goals often conflicted. The key solution was to find a strategy with which both goals could be accomplished. The item selection strategies for graded response model in this study included: the average of the difficulty orders matching with the ability; the medium of the difficulty orders matching with the ability; maximum information; A stratified (average); and A stratified (medium). The evaluation indexes used for comparison included: the bias of ability estimates for the true; the standard error of ability estimates; the average items which the examinees have administered; the standard deviation of the frequency of items selected; and sum of the indices weighted. Using the Monte Carlo simulation method, we obtained some data and computer iterated the data 20 times each under the conditions that the item difficulty parameters followed the normal distribution and even distribution. The results were as follows; The results indicated that no matter difficulty parameters followed the normal distribution or even distribution. Every type of item selection strategies designed in this research had its strong and weak points. In general evaluation, under the condition that items were stratified appropriately, A stratified (medium) (ASM) had the best effect. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aCAT10acomputerized adaptive testing10agraded response model10aitem selection strategies10amultiple choice items1 aHai-qi, D1 aDe-zhi, C1 aShuliang, D1 aTaiping, D uhttp://iacat.org/content/comparison-among-item-selection-strategies-cat-multiple-choice-items03379nas a2200205 4500008004100000020002200041245009700063210006900160260002500229300001200254490000700266520266000273653003402933653002802967100001502995700001903010700001703029700001403046856011303060 2006 eng d a0439-755X (Print)00a[Item Selection Strategies of Computerized Adaptive Testing based on Graded Response Model.]0 aItem Selection Strategies of Computerized Adaptive Testing based bScience Press: China a461-4670 v383 aItem selection strategy (ISS) is an important component of Computerized Adaptive Testing (CAT). Its performance directly affects the security, efficiency and precision of the test. Thus, ISS becomes one of the central issues in CATs based on the Graded Response Model (GRM). It is well known that the goal of IIS is to administer the next unused item remaining in the item bank that best fits the examinees current ability estimate. In dichotomous IRT models, every item has only one difficulty parameter and the item whose difficulty matches the examinee's current ability estimate is considered to be the best fitting item. However, in GRM, each item has more than two ordered categories and has no single value to represent the item difficulty. Consequently, some researchers have used to employ the average or the median difficulty value across categories as the difficulty estimate for the item. Using the average value and the median value in effect introduced two corresponding ISSs. In this study, we used computer simulation compare four ISSs based on GRM. We also discussed the effect of "shadow pool" on the uniformity of pool usage as well as the influence of different item parameter distributions and different ability estimation methods on the evaluation criteria of CAT. In the simulation process, Monte Carlo method was adopted to simulate the entire CAT process; 1,000 examinees drawn from standard normal distribution and four 1,000-sized item pools of different item parameter distributions were also simulated. The assumption of the simulation is that a polytomous item is comprised of six ordered categories. In addition, ability estimates were derived using two methods. They were expected a posteriori Bayesian (EAP) and maximum likelihood estimation (MLE). In MLE, the Newton-Raphson iteration method and the Fisher Score iteration method were employed, respectively, to solve the likelihood equation. Moreover, the CAT process was simulated with each examinee 30 times to eliminate random error. The IISs were evaluated by four indices usually used in CAT from four aspects--the accuracy of ability estimation, the stability of IIS, the usage of item pool, and the test efficiency. Simulation results showed adequate evaluation of the ISS that matched the estimate of an examinee's current trait level with the difficulty values across categories. Setting "shadow pool" in ISS was able to improve the uniformity of pool utilization. Finally, different distributions of the item parameter and different ability estimation methods affected the evaluation indices of CAT. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive testing10aitem selection strategy1 aPing, Chen1 aShuliang, Ding1 aHaijing, Lin1 aJie, Zhou uhttp://iacat.org/content/item-selection-strategies-computerized-adaptive-testing-based-graded-response-model02563nas a2200349 4500008004100000020002200041245016600063210006900229250001500298260000800313300001100321490000700332520142900339653002701768653001601795653001501811653001001826653002101836653001401857653005201871653001501923653001101938653001101949653003701960653001801997653001402015100001502029700001002044700001602054700002502070856011802095 2006 eng d a0003-9993 (Print)00aMeasurement precision and efficiency of multidimensional computer adaptive testing of physical functioning using the pediatric evaluation of disability inventory0 aMeasurement precision and efficiency of multidimensional compute a2006/08/29 cSep a1223-90 v873 aOBJECTIVE: To compare the measurement efficiency and precision of a multidimensional computer adaptive testing (M-CAT) application to a unidimensional CAT (U-CAT) comparison using item bank data from 2 of the functional skills scales of the Pediatric Evaluation of Disability Inventory (PEDI). DESIGN: Using existing PEDI mobility and self-care item banks, we compared the stability of item calibrations and model fit between unidimensional and multidimensional Rasch models and compared the efficiency and precision of the U-CAT- and M-CAT-simulated assessments to a random draw of items. SETTING: Pediatric rehabilitation hospital and clinics. PARTICIPANTS: Clinical and normative samples. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Not applicable. RESULTS: The M-CAT had greater levels of precision and efficiency than the separate mobility and self-care U-CAT versions when using a similar number of items for each PEDI subdomain. Equivalent estimation of mobility and self-care scores can be achieved with a 25% to 40% item reduction with the M-CAT compared with the U-CAT. CONCLUSIONS: M-CAT applications appear to have both precision and efficiency advantages compared with separate U-CAT assessments when content subdomains have a high correlation. Practitioners may also realize interpretive advantages of reporting test score information for each subdomain when separate clinical inferences are desired.10a*Disability Evaluation10a*Pediatrics10aAdolescent10aChild10aChild, Preschool10aComputers10aDisabled Persons/*classification/rehabilitation10aEfficiency10aHumans10aInfant10aOutcome Assessment (Health Care)10aPsychometrics10aSelf Care1 aHaley, S M1 aNi, P1 aLudlow, L H1 aFragala-Pinkham, M A uhttp://iacat.org/content/measurement-precision-and-efficiency-multidimensional-computer-adaptive-testing-physical01617nas a2200217 4500008004100000020002200041245008200063210006900145260002600214300001200240490000700252520088700259653002801146653002501174653002501199653001901224653001601243100001601259700002501275856009901300 2006 eng d a0146-6216 (Print)00aOptimal testing with easy or difficult items in computerized adaptive testing0 aOptimal testing with easy or difficult items in computerized ada bSage Publications: US a379-3930 v303 aComputerized adaptive tests (CATs) are individualized tests that, from a measurement point of view, are optimal for each individual, possibly under some practical conditions. In the present study, it is shown that maximum information item selection in CATs using an item bank that is calibrated with the one- or the two-parameter logistic model results in each individual answering about 50% of the items correctly. Two item selection procedures giving easier (or more difficult) tests for students are presented and evaluated. Item selection on probability points of items yields good results only with the one-parameter logistic model and not with the two-parameter logistic model. An alternative selection procedure, based on maximum information at a shifted ability level, gives satisfactory results with both models. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputer adaptive tests10aindividualized tests10aItem Response Theory10aitem selection10aMeasurement1 aEggen, Theo1 aVerschoor, Angela, J uhttp://iacat.org/content/optimal-testing-easy-or-difficult-items-computerized-adaptive-testing02107nas a2200229 4500008004100000245013800041210006900179300001400248490000700262520127000269653003101539653003401570653002501604653001701629653001901646653002401665100001401689700001801703700001701721700001901738856012001757 2006 eng d00aSimulated computerized adaptive test for patients with lumbar spine impairments was efficient and produced valid measures of function0 aSimulated computerized adaptive test for patients with lumbar sp a947–9560 v593 aObjective: To equate physical functioning (PF) items with Back Pain Functional Scale (BPFS) items, develop a computerized adaptive test (CAT) designed to assess lumbar spine functional status (LFS) in people with lumbar spine impairments, and compare discriminant validity of LFS measures (qIRT) generated using all items analyzed with a rating scale Item Response Theory model (RSM) and measures generated using the simulated CAT (qCAT). Methods: We performed a secondary analysis of retrospective intake rehabilitation data. Results: Unidimensionality and local independence of 25 BPFS and PF items were supported. Differential item functioning was negligible for levels of symptom acuity, gender, age, and surgical history. The RSM fit the data well. A lumbar spine specific CAT was developed that was 72% more efficient than using all 25 items to estimate LFS measures. qIRT and qCAT measures did not discriminate patients by symptom acuity, age, or gender, but discriminated patients by surgical history in similar clinically logical ways. qCAT measures were as precise as qIRT measures. Conclusion: A body part specific simulated CAT developed from an LFS item bank was efficient and produced precise measures of LFS without eroding discriminant validity.10aBack Pain Functional Scale10acomputerized adaptive testing10aItem Response Theory10aLumbar spine10aRehabilitation10aTrue-score equating1 aHart, D L1 aMioduski, J E1 aWerneke, M W1 aStratford, P W uhttp://iacat.org/content/simulated-computerized-adaptive-test-patients-lumbar-spine-impairments-was-efficient-and-002068nas a2200217 4500008004500000245013400045210006900179300001200248490000700260520127300267653003401540653004201574653002501616653001901641100001401660700001301674700001801687700001401705700001501719856011601734 2006 Engldsh 00aSimulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function0 aSimulated computerized adaptive test for patients with shoulder  a290-2980 v593 a

Background and Objective: To test unidimensionality and local independence of a set of shoulder functional status (SFS) items,
develop a computerized adaptive test (CAT) of the items using a rating scale item response theory model (RSM), and compare discriminant validity of measures generated using all items (qIRT) and measures generated using the simulated CAT (qCAT).
Study Design and Setting: We performed a secondary analysis of data collected prospectively during rehabilitation of 400 patients
with shoulder impairments who completed 60 SFS items.
Results: Factor analytic techniques supported that the 42 SFS items formed a unidimensional scale and were locally independent. Except for five items, which were deleted, the RSM fit the data well. The remaining 37 SFS items were used to generate the CAT. On average, 6 items on were needed to estimate precise measures of function using the SFS CAT, compared with all 37 SFS items. The qIRT and qCAT measures were highly correlated (r 5 .96) and resulted in similar classifications of patients.
Conclusion: The simulated SFS CAT was efficient and produced precise, clinically relevant measures of functional status with good
discriminating ability. 

10acomputerized adaptive testing10aFlexilevel Scale of Shoulder Function10aItem Response Theory10aRehabilitation1 aHart, D L1 aCook, KF1 aMioduski, J E1 aTeal, C R1 aCrane, P K uhttp://iacat.org/content/simulated-computerized-adaptive-test-patients-shoulder-impairments-was-efficient-and-002247nas a2200301 4500008004500000020001400045245009800059210007100157300001200228490000700240520127200247653003201519653002601551653002801577653001801605653002501623653001601648653001201664653001501676653001801691653001601709653001801725653002701743653001101770100001901781700001601800856012901816 2006 Spandsh  a0212-972800aTécnicas para detectar patrones de respuesta atípicos [Aberrant patterns detection methods]0 aTécnicas para detectar patrones de respuesta atípicos Aberrant p a143-1540 v223 aLa identificación de patrones de respuesta atípicos es de gran utilidad para la construcción de tests y de bancos de ítems con propiedades psicométricas así como para el análisis de validez de los mismos. En este trabajo de revisión se han recogido los más relevantes y novedosos métodos de ajuste de personas que se han elaborado dentro de cada uno de los principales ámbitos de trabajo de la Psicometría: el escalograma de Guttman, la Teoría Clásica de Tests (TCT), la Teoría de la Generalizabilidad (TG), la Teoría de Respuesta al Ítem (TRI), los Modelos de Respuesta al Ítem No Paramétricos (MRINP), los Modelos de Clase Latente de Orden Restringido (MCL-OR) y el Análisis de Estructura de Covarianzas (AEC).Aberrant patterns detection has a great usefulness in order to make tests and item banks with psychometric characteristics and validity analysis of tests and items. The most relevant and newest person-fit methods have been reviewed. All of them have been made in each one of main areas of Psychometry: Guttman's scalogram, Classical Test Theory (CTT), Generalizability Theory (GT), Item Response Theory (IRT), Non-parametric Response Models (NPRM), Order-Restricted Latent Class Models (OR-LCM) and Covariance Structure Analysis (CSA).10aaberrant patterns detection10aClassical Test Theory10ageneralizability theory10aItem Response10aItem Response Theory10aMathematics10amethods10aperson-fit10aPsychometrics10apsychometry10aTest Validity10atest validity analysis10aTheory1 aNúñez, R M N1 aPina, J A L uhttp://iacat.org/content/t%C3%A9cnicas-para-detectar-patrones-de-respuesta-at%C3%ADpicos-aberrant-patterns-detection-methods01465nas a2200217 4500008004100000245015400041210006900195260004600264300001200310520063100322653003000953653001100983653002500994653001601019653002201035653002301057100001801080700001501098700001401113856012001127 2005 eng d00aApplications of item response theory to improve health outcomes assessment: Developing item banks, linking instruments, and computer-adaptive testing0 aApplications of item response theory to improve health outcomes  aCambridge, UKbCambridge University Press a445-4643 a(From the chapter) The current chapter builds on Reise's introduction to the basic concepts, assumptions, popular models, and important features of IRT and discusses the applications of item response theory (IRT) modeling to health outcomes assessment. In particular, we highlight the critical role of IRT modeling in: developing an instrument to match a study's population; linking two or more instruments measuring similar constructs on a common metric; and creating item banks that provide the foundation for tailored short-form instruments or for computerized adaptive assessments. (PsycINFO Database Record (c) 2005 APA )10aComputer Assisted Testing10aHealth10aItem Response Theory10aMeasurement10aTest Construction10aTreatment Outcomes1 aHambleton, RK1 aGotay, C C1 aSnyder, C uhttp://iacat.org/content/applications-item-response-theory-improve-health-outcomes-assessment-developing-item-banks03119nas a2200385 4500008004100000020002200041245012900063210006900192250001500261260000800276300001000284490000700294520188400301653002502185653002702210653001502237653001002252653002102262653002802283653003802311653001102349653001102360653001102371653000902382653004602391653002702437653003002464653003202494100001502526700001602541700001602557700001502573700002502588856012002613 2005 eng d a0003-9993 (Print)00aAssessing mobility in children using a computer adaptive testing version of the pediatric evaluation of disability inventory0 aAssessing mobility in children using a computer adaptive testing a2005/05/17 cMay a932-90 v863 aOBJECTIVE: To assess score agreement, validity, precision, and response burden of a prototype computerized adaptive testing (CAT) version of the Mobility Functional Skills Scale (Mob-CAT) of the Pediatric Evaluation of Disability Inventory (PEDI) as compared with the full 59-item version (Mob-59). DESIGN: Computer simulation analysis of cross-sectional and longitudinal retrospective data; and cross-sectional prospective study. SETTING: Pediatric rehabilitation hospital, including inpatient acute rehabilitation, day school program, outpatient clinics, community-based day care, preschool, and children's homes. PARTICIPANTS: Four hundred sixty-nine children with disabilities and 412 children with no disabilities (analytic sample); 41 children without disabilities and 39 with disabilities (cross-validation sample). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Summary scores from a prototype Mob-CAT application and versions using 15-, 10-, and 5-item stopping rules; scores from the Mob-59; and number of items and time (in seconds) to administer assessments. RESULTS: Mob-CAT scores from both computer simulations (intraclass correlation coefficient [ICC] range, .94-.99) and field administrations (ICC=.98) were in high agreement with scores from the Mob-59. Using computer simulations of retrospective data, discriminant validity, and sensitivity to change of the Mob-CAT closely approximated that of the Mob-59, especially when using the 15- and 10-item stopping rule versions of the Mob-CAT. The Mob-CAT used no more than 15% of the items for any single administration, and required 20% of the time needed to administer the Mob-59. CONCLUSIONS: Comparable score estimates for the PEDI mobility scale can be obtained from CAT administrations, with losses in validity and precision for shorter forms, but with a considerable reduction in administration time.10a*Computer Simulation10a*Disability Evaluation10aAdolescent10aChild10aChild, Preschool10aCross-Sectional Studies10aDisabled Children/*rehabilitation10aFemale10aHumans10aInfant10aMale10aOutcome Assessment (Health Care)/*methods10aRehabilitation Centers10aRehabilitation/*standards10aSensitivity and Specificity1 aHaley, S M1 aRaczek, A E1 aCoster, W J1 aDumas, H M1 aFragala-Pinkham, M A uhttp://iacat.org/content/assessing-mobility-children-using-computer-adaptive-testing-version-pediatric-evaluation-001369nas a2200193 4500008004100000245005000041210004700091300001000138490000600148520081200154653001700966653002100983653002601004653002501030100001501055700001401070700002301084856006801107 2005 eng d00aAn Authoring Environment for Adaptive Testing0 aAuthoring Environment for Adaptive Testing a66-760 v83 a

SIETTE is a web-based adaptive testing system. It implements Computerized Adaptive Tests. These tests are tailor-made, theory-based tests, where questions shown to students, finalization of the test, and student knowledge estimation is accomplished adaptively. To construct these tests, SIETTE has an authoring environment comprising a suite of tools that helps teachers create questions and tests properly, and analyze students’ performance after taking a test. In this paper, we present this authoring environment in the
framework of adaptive testing. As will be shown, this set of visual tools, that contain some adaptable eatures, can be useful for teachers lacking skills in this kind of testing. Additionally, other systems that implement adaptive testing will be studied. 

10aAdaptability10aAdaptive Testing10aAuthoring environment10aItem Response Theory1 aGuzmán, E1 aConejo, R1 aGarcía-Hervás, E uhttp://iacat.org/content/authoring-environment-adaptive-testing02238nas a2200205 4500008004100000020004600041245009500087210006900182260002700251300001200278490000700290520149400297653002701791653003001818653001701848653002501865100001901890700001001909856011301919 2005 eng d a1560-4292 (Print); 1560-4306 (Electronic)00aA Bayesian student model without hidden nodes and its comparison with item response theory0 aBayesian student model without hidden nodes and its comparison w bIOS Press: Netherlands a291-3230 v153 aThe Bayesian framework offers a number of techniques for inferring an individual's knowledge state from evidence of mastery of concepts or skills. A typical application where such a technique can be useful is Computer Adaptive Testing (CAT). A Bayesian modeling scheme, POKS, is proposed and compared to the traditional Item Response Theory (IRT), which has been the prevalent CAT approach for the last three decades. POKS is based on the theory of knowledge spaces and constructs item-to-item graph structures without hidden nodes. It aims to offer an effective knowledge assessment method with an efficient algorithm for learning the graph structure from data. We review the different Bayesian approaches to modeling student ability assessment and discuss how POKS relates to them. The performance of POKS is compared to the IRT two parameter logistic model. Experimental results over a 34 item Unix test and a 160 item French language test show that both approaches can classify examinees as master or non-master effectively and efficiently, with relatively comparable performance. However, more significant differences are found in favor of POKS for a second task that consists in predicting individual question item outcome. Implications of these results for adaptive testing and student modeling are discussed, as well as the limitations and advantages of POKS, namely the issue of integrating concepts into its structure. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aBayesian Student Model10acomputer adaptive testing10ahidden nodes10aItem Response Theory1 aDesmarais, M C1 aPu, X uhttp://iacat.org/content/bayesian-student-model-without-hidden-nodes-and-its-comparison-item-response-theory02148nas a2200229 4500008004100000020002200041245008700063210006900150260004100219300001200260490000700272520135500279653002101634653001501655653002401670653002601694653002501720653002101745653003101766100002301797856009801820 2005 eng d a0022-0655 (Print)00aA comparison of item-selection methods for adaptive tests with content constraints0 acomparison of itemselection methods for adaptive tests with cont bBlackwell Publishing: United Kingdom a283-3020 v423 aIn test assembly, a fundamental difference exists between algorithms that select a test sequentially or simultaneously. Sequential assembly allows us to optimize an objective function at the examinee's ability estimate, such as the test information function in computerized adaptive testing. But it leads to the non-trivial problem of how to realize a set of content constraints on the test—a problem more naturally solved by a simultaneous item-selection method. Three main item-selection methods in adaptive testing offer solutions to this dilemma. The spiraling method moves item selection across categories of items in the pool proportionally to the numbers needed from them. Item selection by the weighted-deviations method (WDM) and the shadow test approach (STA) is based on projections of the future consequences of selecting an item. These two methods differ in that the former calculates a projection of a weighted sum of the attributes of the eventual test and the latter a projection of the test itself. The pros and cons of these methods are analyzed. An empirical comparison between the WDM and STA was conducted for an adaptive version of the Law School Admission Test (LSAT), which showed equally good item-exposure rates but violations of some of the constraints and larger bias and inaccuracy of the ability estimator for the WDM.10aAdaptive Testing10aAlgorithms10acontent constraints10aitem selection method10ashadow test approach10aspiraling method10aweighted deviations method1 avan der Linden, WJ uhttp://iacat.org/content/comparison-item-selection-methods-adaptive-tests-content-constraints02787nas a2200469 4500008004100000020002200041245010400063210006900167250001500236260000800251300001200259490000700271520132800278653002201606653003101628653001501659653001601674653001001690653003401700653002101734653002401755653002501779653001501804653001101819653005301830653002901883653001101912653001101923653002001934653000901954653003101963653004601994653003102040653001402071653003202085100001502117700001002132700002502142700001702167700001302184856012002197 2005 eng d a0012-1622 (Print)00aA computer adaptive testing approach for assessing physical functioning in children and adolescents0 acomputer adaptive testing approach for assessing physical functi a2005/02/15 cFeb a113-1200 v473 aThe purpose of this article is to demonstrate: (1) the accuracy and (2) the reduction in amount of time and effort in assessing physical functioning (self-care and mobility domains) of children and adolescents using computer-adaptive testing (CAT). A CAT algorithm selects questions directly tailored to the child's ability level, based on previous responses. Using a CAT algorithm, a simulation study was used to determine the number of items necessary to approximate the score of a full-length assessment. We built simulated CAT (5-, 10-, 15-, and 20-item versions) for self-care and mobility domains and tested their accuracy in a normative sample (n=373; 190 males, 183 females; mean age 6y 11mo [SD 4y 2m], range 4mo to 14y 11mo) and a sample of children and adolescents with Pompe disease (n=26; 21 males, 5 females; mean age 6y 1mo [SD 3y 10mo], range 5mo to 14y 10mo). Results indicated that comparable score estimates (based on computer simulations) to the full-length tests can be achieved in a 20-item CAT version for all age ranges and for normative and clinical samples. No more than 13 to 16% of the items in the full-length tests were needed for any one administration. These results support further consideration of using CAT programs for accurate and efficient clinical assessments of physical functioning.10a*Computer Systems10aActivities of Daily Living10aAdolescent10aAge Factors10aChild10aChild Development/*physiology10aChild, Preschool10aComputer Simulation10aConfidence Intervals10aDemography10aFemale10aGlycogen Storage Disease Type II/physiopathology10aHealth Status Indicators10aHumans10aInfant10aInfant, Newborn10aMale10aMotor Activity/*physiology10aOutcome Assessment (Health Care)/*methods10aReproducibility of Results10aSelf Care10aSensitivity and Specificity1 aHaley, S M1 aNi, P1 aFragala-Pinkham, M A1 aSkrinar, A M1 aCorzo, D uhttp://iacat.org/content/computer-adaptive-testing-approach-assessing-physical-functioning-children-and-adolescents01555nas a2200169 4500008004100000245008000041210006900121300001200190490000700202520094200209653002101151653003001172653005401202100001401256700001301270856010201283 2005 eng d00aControlling item exposure and test overlap in computerized adaptive testing0 aControlling item exposure and test overlap in computerized adapt a204-2170 v293 aThis article proposes an item exposure control method, which is the extension of the Sympson and Hetter procedure and can provide item exposure control at both the item and test levels. Item exposure rate and test overlap rate are two indices commonly used to track item exposure in computerized adaptive tests. By considering both indices, item exposure can be monitored at both the item and test levels. To control the item exposure rate and test overlap rate simultaneously, the modified procedure attempted to control not only the maximum value but also the variance of item exposure rates. Results indicated that the item exposure rate and test overlap rate could be controlled simultaneously by implementing the modified procedure. Item exposure control was improved and precision of trait estimation decreased when a prespecified maximum test overlap rate was stringent. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Content (Test) computerized adaptive testing1 aChen, S-Y1 aLei, P-W uhttp://iacat.org/content/controlling-item-exposure-and-test-overlap-computerized-adaptive-testing02210nas a2200373 4500008004100000245011500041210006900156300001100225490000700236520104400243653002101287653002001308653001001328653000901338653002801347653001101375653003801386653000901424653001601433653004101449653001801490653003701508653003001545100001401575700001301589700001301602700001501615700001701630700001501647700001901662700001601681700001801697856012101715 2005 eng d00aData pooling and analysis to build a preliminary item bank: an example using bowel function in prostate cancer0 aData pooling and analysis to build a preliminary item bank an ex a142-590 v283 aAssessing bowel function (BF) in prostate cancer can help determine therapeutic trade-offs. We determined the components of BF commonly assessed in prostate cancer studies as an initial step in creating an item bank for clinical and research application. We analyzed six archived data sets representing 4,246 men with prostate cancer. Thirty-one items from validated instruments were available for analysis. Items were classified into domains (diarrhea, rectal urgency, pain, bleeding, bother/distress, and other) then subjected to conventional psychometric and item response theory (IRT) analyses. Items fit the IRT model if the ratio between observed and expected item variance was between 0.60 and 1.40. Four of 31 items had inadequate fit in at least one analysis. Poorly fitting items included bleeding (2), rectal urgency (1), and bother/distress (1). A fifth item assessing hemorrhoids was poorly correlated with other items. Our analyses supported four related components of BF: diarrhea, rectal urgency, pain, and bother/distress.10a*Quality of Life10a*Questionnaires10aAdult10aAged10aData Collection/methods10aHumans10aIntestine, Large/*physiopathology10aMale10aMiddle Aged10aProstatic Neoplasms/*physiopathology10aPsychometrics10aResearch Support, Non-U.S. Gov't10aStatistics, Nonparametric1 aEton, D T1 aLai, J S1 aCella, D1 aReeve, B B1 aTalcott, J A1 aClark, J A1 aMcPherson, C P1 aLitwin, M S1 aMoinpour, C M uhttp://iacat.org/content/data-pooling-and-analysis-build-preliminary-item-bank-example-using-bowel-function-prostate01996nas a2200205 4500008004100000020004600041245007900087210006900166260004100235300001400276490000700290520127200297653003001569653002501599653003401624100001301658700001801671700001601689856008501705 2005 eng d a0017-9124 (Print); 1475-6773 (Electronic)00aDynamic assessment of health outcomes: Time to let the CAT out of the bag?0 aDynamic assessment of health outcomes Time to let the CAT out of bBlackwell Publishing: United Kingdom a1694-17110 v403 aBackground: The use of item response theory (IRT) to measure self-reported outcomes has burgeoned in recent years. Perhaps the most important application of IRT is computer-adaptive testing (CAT), a measurement approach in which the selection of items is tailored for each respondent. Objective. To provide an introduction to the use of CAT in the measurement of health outcomes, describe several IRT models that can be used as the basis of CAT, and discuss practical issues associated with the use of adaptive scaling in research settings. Principal Points: The development of a CAT requires several steps that are not required in the development of a traditional measure including identification of "starting" and "stopping" rules. CAT's most attractive advantage is its efficiency. Greater measurement precision can be achieved with fewer items. Disadvantages of CAT include the high cost and level of technical expertise required to develop a CAT. Conclusions: Researchers, clinicians, and patients benefit from the availability of psychometrically rigorous measures that are not burdensome. CAT outcome measures hold substantial promise in this regard, but their development is not without challenges. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputer adaptive testing10aItem Response Theory10aself reported health outcomes1 aCook, KF1 aO'Malley, K J1 aRoddey, T S uhttp://iacat.org/content/dynamic-assessment-health-outcomes-time-let-cat-out-bag02237nas a2200217 4500008004100000020002200041245014200063210006900205260004100274300001200315490000700327520142600334653001401760653003401774653002301808653001601831653002701847100001201874700001701886856011601903 2005 eng d a0022-0655 (Print)00aIncreasing the homogeneity of CAT's item-exposure rates by minimizing or maximizing varied target functions while assembling shadow tests0 aIncreasing the homogeneity of CATs itemexposure rates by minimiz bBlackwell Publishing: United Kingdom a245-2690 v423 aA computerized adaptive testing (CAT) algorithm that has the potential to increase the homogeneity of CATs item-exposure rates without significantly sacrificing the precision of ability estimates was proposed and assessed in the shadow-test (van der Linden & Reese, 1998) CAT context. This CAT algorithm was formed by a combination of maximizing or minimizing varied target functions while assembling shadow tests. There were four target functions to be separately used in the first, second, third, and fourth quarter test of CAT. The elements to be used in the four functions were associated with (a) a random number assigned to each item, (b) the absolute difference between an examinee's current ability estimate and an item difficulty, (c) the absolute difference between an examinee's current ability estimate and an optimum item difficulty, and (d) item information. The results indicated that this combined CAT fully utilized all the items in the pool, reduced the maximum exposure rates, and achieved more homogeneous exposure rates. Moreover, its precision in recovering ability estimates was similar to that of the maximum item-information method. The combined CAT method resulted in the best overall results compared with the other individual CAT item-selection methods. The findings from the combined CAT are encouraging. Future uses are discussed. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aalgorithm10acomputerized adaptive testing10aitem exposure rate10ashadow test10avaried target function1 aLi, Y H1 aSchafer, W D uhttp://iacat.org/content/increasing-homogeneity-cats-item-exposure-rates-minimizing-or-maximizing-varied-target02016nas a2200193 4500008004100000245009300041210006900134300001200203490000700215520136400222653001501586653002401601653001101625653002201636100001801658700001801676700001901694856010901713 2005 eng d00aInfeasibility in automated test assembly models: A comparison study of different methods0 aInfeasibility in automated test assembly models A comparison stu a223-2430 v423 aSeveral techniques exist to automatically put together a test meeting a number of specifications. In an item bank, the items are stored with their characteristics. A test is constructed by selecting a set of items that fulfills the specifications set by the test assembler. Test assembly problems are often formulated in terms of a model consisting of restrictions and an objective to be maximized or minimized. A problem arises when it is impossible to construct a test from the item pool that meets all specifications, that is, when the model is not feasible. Several methods exist to handle these infeasibility problems. In this article, test assembly models resulting from two practical testing programs were reconstructed to be infeasible. These models were analyzed using methods that forced a solution (Goal Programming, Multiple-Goal Programming, Greedy Heuristic), that analyzed the causes (Relaxed and Ordered Deletion Algorithm (RODA), Integer Randomized Deletion Algorithm (IRDA), Set Covering (SC), and Item Sampling), or that analyzed the causes and used this information to force a solution (Irreducible Infeasible Set-Solver). Specialized methods such as the IRDA and the Irreducible Infeasible Set-Solver performed best. Recommendations about the use of different methods are given. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAlgorithms10aItem Content (Test)10aModels10aTest Construction1 aHuitzing, H A1 aVeldkamp, B P1 aVerschoor, A J uhttp://iacat.org/content/infeasibility-automated-test-assembly-models-comparison-study-different-methods01311nas a2200229 4500008004100000020002200041245007700063210006900140260002500209300001200234490000700246520057900253653002700832653003000859653002500889100001100914700001600925700001500941700001300956700001500969856009700984 2005 eng d a0439-755X (Print)00a[Item characteristic curve equating under graded response models in IRT]0 aItem characteristic curve equating under graded response models  bScience Press: China a832-8380 v373 aIn one of the largest qualificatory tests--economist test, to guarantee the comparability among different years, construct item bank and prepare for computerized adaptive testing, item characteristic curve equating and anchor test equating design under graded models in IRT are used, which have realized the item and ability parameter equating of test data in five years and succeeded in establishing an item bank. Based on it, cut scores of different years are compared by equating and provide demonstrational gist to constitute the eligibility standard of economist test. 10agraded response models10aitem characteristic curve10aItem Response Theory1 aJun, Z1 aDongming, O1 aShuyuan, X1 aHaiqi, D1 aShuqing, Q uhttp://iacat.org/content/item-characteristic-curve-equating-under-graded-response-models-irt02850nas a2200241 4500008004100000245018600041210007000227300001200297490000700309520195000316653003002266653002502296653001802321653002502339653001802364653001802382653001102400100001402411700001502425700001902440700002002459856012902479 2005 eng d00aPropiedades psicométricas de un test Adaptativo Informatizado para la medición del ajuste emocional [Psychometric properties of an Emotional Adjustment Computerized Adaptive Test]0 aPropiedades psicométricas de un test Adaptativo Informatizado pa a484-4910 v173 aEn el presente trabajo se describen las propiedades psicométricas de un Test Adaptativo Informatizado para la medición del ajuste emocional de las personas. La revisión de la literatura acerca de la aplicación de los modelos de la teoría de la respuesta a los ítems (TRI) muestra que ésta se ha utilizado más en el trabajo con variables aptitudinales que para la medición de variables de personalidad, sin embargo diversos estudios han mostrado la eficacia de la TRI para la descripción psicométrica de dichasvariables. Aun así, pocos trabajos han explorado las características de un Test Adaptativo Informatizado, basado en la TRI, para la medición de una variable de personalidad como es el ajuste emocional. Nuestros resultados muestran la eficiencia del TAI para la evaluación del ajuste emocional, proporcionando una medición válida y precisa, utilizando menor número de elementos de medida encomparación con las escalas de ajuste emocional de instrumentos fuertemente implantados. Psychometric properties of an emotional adjustment computerized adaptive test. In the present work it was described the psychometric properties of an emotional adjustment computerized adaptive test. An examination of Item Response Theory (IRT) research literature indicates that IRT has been mainly used for assessing achievements and ability rather than personality factors. Nevertheless last years have shown several studies wich have successfully used IRT to personality assessment instruments. Even so, a few amount of works has inquired the computerized adaptative test features, based on IRT, for the measurement of a personality traits as it’s the emotional adjustment. Our results show the CAT efficiency for the emotional adjustment assessment so this provides a valid and accurate measurement; by using a less number of items in comparison with the emotional adjustment scales from the most strongly established questionnaires.10aComputer Assisted Testing10aEmotional Adjustment10aItem Response10aPersonality Measures10aPsychometrics10aTest Validity10aTheory1 aAguado, D1 aRubio, V J1 aHontangas, P M1 aHernández, J M uhttp://iacat.org/content/propiedades-psicom%C3%A9tricas-de-un-test-adaptativo-informatizado-para-la-medici%C3%B3n-del-ajuste01472nas a2200193 4500008004100000245020200041210006900243300001200312490000700324520066400331653002100995653003001016653005501046653001101101653001801112100001401130700001701144856011701161 2005 eng d00aSomministrazione di test computerizzati di tipo adattivo: Un' applicazione del modello di misurazione di Rasch [Administration of computerized and adaptive tests: An application of the Rasch Model]0 aSomministrazione di test computerizzati di tipo adattivo Un appl a131-1490 v123 aThe aim of the present study is to describe the characteristics of a procedure for administering computerized and adaptive tests (Computer Adaptive Testing or CAT). Items to be asked to the individuals are interactively chosen and are selected from a "bank" in which they were previously calibrated and recorded on the basis of their difficulty level. The selection of items is performed by increasingly more accurate estimates of the examinees' ability. The building of an item-bank on Psychometrics and the implementation of this procedure allow a first validation through Monte Carlo simulations. (PsycINFO Database Record (c) 2006 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Response Theory computerized adaptive testing10aModels10aPsychometrics1 aMiceli, R1 aMolinengo, G uhttp://iacat.org/content/somministrazione-di-test-computerizzati-di-tipo-adattivo-un-applicazione-del-modello-di01468nas a2200217 4500008004100000245004600041210004600087300001200133490000700145520085100152653001801003653002501021653003201046653001301078653002201091653002401113653001501137100001601152700001501168856006701183 2005 eng d00aTest construction for cognitive diagnosis0 aTest construction for cognitive diagnosis a262-2770 v293 aAlthough cognitive diagnostic models (CDMs) can be useful in the analysis and interpretation of existing tests, little has been developed to specify how one might construct a good test using aspects of the CDMs. This article discusses the derivation of a general CDM index based on Kullback-Leibler information that will serve as a measure of how informative an item is for the classification of examinees. The effectiveness of the index is examined for items calibrated using the deterministic input noisy "and" gate model (DINA) and the reparameterized unified model (RUM) by implementing a simple heuristic to construct a test from an item bank. When compared to randomly constructed tests from the same item bank, the heuristic shows significant improvement in classification rates. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10a(Measurement)10aCognitive Assessment10aItem Analysis (Statistical)10aProfiles10aTest Construction10aTest Interpretation10aTest Items1 aHenson, R K1 aDouglas, J uhttp://iacat.org/content/test-construction-cognitive-diagnosis01880nas a2200241 4500008004100000245006000041210005900101260004800160300001200208520112200220653001501342653003401357653002201391653002101413653001901434653001601453653001701469653001301486653002801499100001301527700001601540856008201556 2004 eng d00aAdaptive computerized educational systems: A case study0 aAdaptive computerized educational systems A case study aSan Diego, CA. USAbElsevier Academic Press a143-1693 a(Created by APA) Adaptive instruction describes adjustments typical of one-on-one tutoring as discussed in the college tutorial scenario. So computerized adaptive instruction refers to the use of computer software--almost always incorporating artificially intelligent services--which has been designed to adjust both the presentation of information and the form of questioning to meet the current needs of an individual learner. This chapter describes a system for Internet-delivered adaptive instruction. The author attempts to demonstrate a sharp difference between the teaching that takes place outside of the classroom in universities and the kind that is at least afforded, if not taken advantage of by many, students in a more personalized educational setting such as those in the small liberal arts colleges. The author describes a computer-based technology that allows that gap to be bridged with the advantage of at least having more highly prepared learners sitting in college classrooms. A limited range of emerging research that supports that proposition is cited. (PsycINFO Database Record (c) 2005 APA )10aArtificial10aComputer Assisted Instruction10aComputer Software10aHigher Education10aIndividualized10aInstruction10aIntelligence10aInternet10aUndergraduate Education1 aRay, R D1 aMalott, R W uhttp://iacat.org/content/adaptive-computerized-educational-systems-case-study02589nas a2200469 4500008004100000245007200041210006900113300001000182490000600192520108600198653002501284653001001309653001501319653002101334653002201355653005901377653007001436653003301506653001101539653001101550653001301561653000901574653002701583653002201610653005501632653001901687653001501706653006601721653001801787653003701805653004101842653003001883653001301913100001501926700001301941700001801954700001501972700001401987700001402001700001302015856009102028 2004 eng d00aComputerized adaptive measurement of depression: A simulation study0 aComputerized adaptive measurement of depression A simulation stu a13-230 v43 aBackground: Efficient, accurate instruments for measuring depression are increasingly importantin clinical practice. We developed a computerized adaptive version of the Beck DepressionInventory (BDI). We examined its efficiency and its usefulness in identifying Major DepressiveEpisodes (MDE) and in measuring depression severity.Methods: Subjects were 744 participants in research studies in which each subject completed boththe BDI and the SCID. In addition, 285 patients completed the Hamilton Depression Rating Scale.Results: The adaptive BDI had an AUC as an indicator of a SCID diagnosis of MDE of 88%,equivalent to the full BDI. The adaptive BDI asked fewer questions than the full BDI (5.6 versus 21items). The adaptive latent depression score correlated r = .92 with the BDI total score and thelatent depression score correlated more highly with the Hamilton (r = .74) than the BDI total scoredid (r = .70).Conclusions: Adaptive testing for depression may provide greatly increased efficiency withoutloss of accuracy in identifying MDE or in measuring depression severity.10a*Computer Simulation10aAdult10aAlgorithms10aArea Under Curve10aComparative Study10aDepressive Disorder/*diagnosis/epidemiology/psychology10aDiagnosis, Computer-Assisted/*methods/statistics & numerical data10aFactor Analysis, Statistical10aFemale10aHumans10aInternet10aMale10aMass Screening/methods10aPatient Selection10aPersonality Inventory/*statistics & numerical data10aPilot Projects10aPrevalence10aPsychiatric Status Rating Scales/*statistics & numerical data10aPsychometrics10aResearch Support, Non-U.S. Gov't10aResearch Support, U.S. Gov't, P.H.S.10aSeverity of Illness Index10aSoftware1 aGardner, W1 aShear, K1 aKelleher, K J1 aPajer, K A1 aMammen, O1 aBuysse, D1 aFrank, E uhttp://iacat.org/content/computerized-adaptive-measurement-depression-simulation-study01904nas a2200217 4500008004100000020004600041245010100087210006900188260002600257300001200283490000700295520111100302653002401413653003201437653001301469653003801482100001701520700001501537700001401552856012001566 2004 eng d a0021-9762 (Print); 1097-4679 (Electronic)00aComputers in clinical assessment: Historical developments, present status, and future challenges0 aComputers in clinical assessment Historical developments present bJohn Wiley & Sons: US a331-3450 v603 aComputerized testing methods have long been regarded as a potentially powerful asset for providing psychological assessment services. Ever since computers were first introduced and adapted to the field of assessment psychology in the 1950s, they have been a valuable aid for scoring, data processing, and even interpretation of test results. The history and status of computer-based personality and neuropsychological tests are discussed in this article. Several pertinent issues involved in providing test interpretation by computer are highlighted. Advances in computer-based test use, such as computerized adaptive testing, are described and problems noted. Today, there is great interest in expanding the availability of psychological assessment applications on the Internet. Although these applications show great promise, there are a number of problems associated with providing psychological tests on the Internet that need to be addressed by psychologists before the Internet can become a major medium for psychological service delivery. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aclinical assessment10acomputerized testing method10aInternet10apsychological assessment services1 aButcher, J N1 aPerry, J L1 aHahn, J A uhttp://iacat.org/content/computers-clinical-assessment-historical-developments-present-status-and-future-challenges01608nas a2200217 4500008004100000020002200041245008200063210006900145260004300214300001200257490000700269520084600276653003401122653002601156653003501182653001601217653001701233100002301250700001801273856009901291 2004 eng d a1076-9986 (Print)00aConstraining item exposure in computerized adaptive testing with shadow tests0 aConstraining item exposure in computerized adaptive testing with bAmerican Educational Research Assn: US a273-2910 v293 aItem-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter’s (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. The method does not, however, require time-consuming simulation studies to set values for control parameters before the operational use of the test. Instead, it can set the probabilities of item ineligibility adaptively during the test using the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test showed that application of the method yielded perfect control of the item-exposure rates and had negligible impact on the bias and mean-squared error functions of the ability estimator. 10acomputerized adaptive testing10aitem exposure control10aitem ineligibility constraints10aProbability10ashadow tests1 avan der Linden, WJ1 aVeldkamp, B P uhttp://iacat.org/content/constraining-item-exposure-computerized-adaptive-testing-shadow-tests01903nas a2200217 4500008004100000020002200041245007000063210006900133260004100202300001200243490000700255520117100262653003201433653003301465653001801498653002401516100001301540700001801553700002301571856009101594 2004 eng d a0022-0655 (Print)00aConstructing rotating item pools for constrained adaptive testing0 aConstructing rotating item pools for constrained adaptive testin bBlackwell Publishing: United Kingdom a345-3590 v413 aPreventing items in adaptive testing from being over- or underexposed is one of the main problems in computerized adaptive testing. Though the problem of overexposed items can be solved using a probabilistic item-exposure control method, such methods are unable to deal with the problem of underexposed items. Using a system of rotating item pools, on the other hand, is a method that potentially solves both problems. In this method, a master pool is divided into (possibly overlapping) smaller item pools, which are required to have similar distributions of content and statistical attributes. These pools are rotated among the testing sites to realize desirable exposure rates for the items. A test assembly model, motivated by Gulliksen's matched random subtests method, was explored to help solve the problem of dividing a master pool into a set of smaller pools. Different methods to solve the model are proposed. An item pool from the Law School Admission Test was used to evaluate the performances of computerized adaptive tests from systems of rotating item pools constructed using these methods. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive tests10aconstrained adaptive testing10aitem exposure10arotating item pools1 aAriel, A1 aVeldkamp, B P1 avan der Linden, WJ uhttp://iacat.org/content/constructing-rotating-item-pools-constrained-adaptive-testing01989nas a2200193 4500008004100000020002200041245011400063210006900177260004100246300001200287490000700299520125600306653003401562653002501596653002601621100001401647700001901661856011501680 2004 eng d a0022-0655 (Print)00aEffects of practical constraints on item selection rules at the early stages of computerized adaptive testing0 aEffects of practical constraints on item selection rules at the  bBlackwell Publishing: United Kingdom a149-1740 v413 aThe purpose of this study was to compare the effects of four item selection rules--(1) Fisher information (F), (2) Fisher information with a posterior distribution (FP), (3) Kullback-Leibler information with a posterior distribution (KP), and (4) completely randomized item selection (RN)--with respect to the precision of trait estimation and the extent of item usage at the early stages of computerized adaptive testing. The comparison of the four item selection rules was carried out under three conditions: (1) using only the item information function as the item selection criterion; (2) using both the item information function and content balancing; and (3) using the item information function, content balancing, and item exposure control. When test length was less than 10 items, FP and KP tended to outperform F at extreme trait levels in Condition 1. However, in more realistic settings, it could not be concluded that FP and KP outperformed F, especially when item exposure control was imposed. When test length was greater than 10 items, the three nonrandom item selection procedures performed similarly no matter what the condition was, while F had slightly higher item usage. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive testing10aitem selection rules10apractical constraints1 aChen, S-Y1 aAnkenmann, R D uhttp://iacat.org/content/effects-practical-constraints-item-selection-rules-early-stages-computerized-adaptive01899nas a2200193 4500008004100000020002200041245010000063210006900163260004300232300001200275490000700287520117500294653002301469653003001492653002301522653002501545100001601570856011901586 2004 eng d a1076-9986 (Print)00aEstimating ability and item-selection strategy in self-adapted testing: A latent class approach0 aEstimating ability and itemselection strategy in selfadapted tes bAmerican Educational Research Assn: US a379-3960 v293 aThis article presents a psychometric model for estimating ability and item-selection strategies in self-adapted testing. In contrast to computer adaptive testing, in self-adapted testing the examinees are allowed to select the difficulty of the items. The item-selection strategy is defined as the distribution of difficulty conditional on the responses given to previous items. The article shows that missing responses in self-adapted testing are missing at random and can be ignored in the estimation of ability. However, the item-selection strategy cannot always be ignored in such an estimation. An EM algorithm is presented to estimate an examinee's ability and strategies, and a model fit is evaluated using Akaike's information criterion. The article includes an application with real data to illustrate how the model can be used in practice for evaluating hypotheses, estimating ability, and identifying strategies. In the example, four strategies were identified and related to examinees' ability. It was shown that individual examinees tended not to follow a consistent strategy throughout the test. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10aestimating ability10aitem-selection strategies10apsychometric model10aself-adapted testing1 aRevuelta, J uhttp://iacat.org/content/estimating-ability-and-item-selection-strategy-self-adapted-testing-latent-class-approach01891nas a2200181 4500008004100000020002200041245012000063210006900183260002600252300001200278490000700290520119300297653003401490653003701524653001801561100001401579856011601593 2004 eng d a0146-6216 (Print)00aStrategies for controlling item exposure in computerized adaptive testing with the generalized partial credit model0 aStrategies for controlling item exposure in computerized adaptiv bSage Publications: US a165-1850 v283 aChoosing a strategy for controlling item exposure has become an integral part of test development for computerized adaptive testing (CAT). This study investigated the performance of six procedures for controlling item exposure in a series of simulated CATs under the generalized partial credit model. In addition to a no-exposure control baseline condition, the randomesque, modified-within-.10-logits, Sympson-Hetter, conditional Sympson-Hetter, a-stratified with multiple-stratification, and enhanced a-stratified with multiple-stratification procedures were implemented to control exposure rates. Two variations of the randomesque and modified-within-.10-logits procedures were examined, which varied the size of the item group from which the next item to be administered was randomly selected. The results indicate that although the conditional Sympson-Hetter provides somewhat lower maximum exposure rates, the randomesque and modified-within-.10-logits procedures with the six-item group variation have great utility for controlling overlap rates and increasing pool utilization and should be given further consideration. (PsycINFO Database Record (c) 2007 APA, all rights reserved)10acomputerized adaptive testing10ageneralized partial credit model10aitem exposure1 aDavis, LL uhttp://iacat.org/content/strategies-controlling-item-exposure-computerized-adaptive-testing-generalized-partial01875nas a2200169 4500008004100000245013100041210006900172300001200241490000700253520122700260653003001487653002501517653001501542653001601557100001601573856011601589 2004 eng d00aUsing patterns of summed scores in paper-and-pencil tests and computer-adaptive tests to detect misfitting item score patterns0 aUsing patterns of summed scores in paperandpencil tests and comp a119-1360 v413 aTwo new methods have been proposed to determine unexpected sum scores on subtests (testlets) both for paper-and-pencil tests and computer adaptive tests. A method based on a conservative bound using the hypergeometric distribution, denoted ρ, was compared with a method where the probability for each score combination was calculated using a highest density region (HDR). Furthermore, these methods were compared with the standardized log-likelihood statistic with and without a correction for the estimated latent trait value (denoted as l-super(*)-sub(z) and l-sub(z), respectively). Data were simulated on the basis of the one-parameter logistic model, and both parametric and nonparametric logistic regression was used to obtain estimates of the latent trait. Results showed that it is important to take the trait level into account when comparing subtest scores. In a nonparametric item response theory (IRT) context, on adapted version of the HDR method was a powerful alterative to ρ. In a parametric IRT context, results showed that l-super(*)-sub(z) had the highest power when the data were simulated conditionally on the estimated latent trait level. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aComputer Assisted Testing10aItem Response Theory10aperson Fit10aTest Scores1 aMeijer, R R uhttp://iacat.org/content/using-patterns-summed-scores-paper-and-pencil-tests-and-computer-adaptive-tests-detect01919nas a2200241 4500008004100000245009400041210006900135300001200204490000700216520110100223653002101324653001301345653003001358653005701388653000901445653003201454653002601486653002001512100001401532700001301546700001501559856010301574 2003 eng d00aA Bayesian method for the detection of item preknowledge in computerized adaptive testing0 aBayesian method for the detection of item preknowledge in comput a121-1370 v273 aWith the increased use of continuous testing in computerized adaptive testing, new concerns about test security have evolved, such as how to ensure that items in an item pool are safeguarded from theft. In this article, procedures to detect test takers using item preknowledge are explored. When test takers use item preknowledge, their item responses deviate from the underlying item response theory (IRT) model, and estimated abilities may be inflated. This deviation may be detected through the use of person-fit indices. A Bayesian posterior log odds ratio index is proposed for detecting the use of item preknowledge. In this approach to person fit, the estimated probability that each test taker has preknowledge of items is updated after each item response. These probabilities are based on the IRT parameters, a model specifying the probability that each item has been memorized, and the test taker's item responses. Simulations based on an operational computerized adaptive test (CAT) pool are used to demonstrate the use of the odds ratio index. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aCheating10aComputer Assisted Testing10aIndividual Differences computerized adaptive testing10aItem10aItem Analysis (Statistical)10aMathematical Modeling10aResponse Theory1 aMcLeod, L1 aLewis, C1 aThissen, D uhttp://iacat.org/content/bayesian-method-detection-item-preknowledge-computerized-adaptive-testing01836nas a2200205 4500008004100000245009000041210006900131300001100200490000700211520111400218653002101332653003001353653001601383653003201399653001601431653004501447100001501492700001601507856010701523 2003 eng d00aA comparative study of item exposure control methods in computerized adaptive testing0 acomparative study of item exposure control methods in computeriz a71-1030 v403 aThis study compared the properties of five methods of item exposure control within the purview of estimating examinees' abilities in a computerized adaptive testing (CAT) context. Each exposure control algorithm was incorporated into the item selection procedure and the adaptive testing progressed based on the CAT design established for this study. The merits and shortcomings of these strategies were considered under different item pool sizes and different desired maximum exposure rates and were evaluated in light of the observed maximum exposure rates, the test overlap rates, and the conditional standard errors of measurement. Each method had its advantages and disadvantages, but no one possessed all of the desired characteristics. There was a clear and logical trade-off between item exposure control and measurement precision. The M. L. Stocking and C. Lewis conditional multinomial procedure and, to a slightly lesser extent, the T. Davey and C. G. Parshall method seemed to be the most promising considering all of the factors that this study addressed. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aComputer Assisted Testing10aEducational10aItem Analysis (Statistical)10aMeasurement10aStrategies computerized adaptive testing1 aChang, S-W1 aAnsley, T N uhttp://iacat.org/content/comparative-study-item-exposure-control-methods-computerized-adaptive-testing01757nas a2200313 4500008004100000245007700041210006900118300001200187490000700199520083400206653002101040653001501061653001701076653001601093653003001109653001701139653001501156653002501171653002001196653001501216653002501231653001801256653000901274100001801283700001201301700001601313700001601329856009801345 2003 eng d00aComputerized adaptive rating scales for measuring managerial performance0 aComputerized adaptive rating scales for measuring managerial per a237-2460 v113 aComputerized adaptive rating scales (CARS) had been developed to measure contextual or citizenship performance. This rating format used a paired-comparison protocol, presenting pairs of behavioral statements scaled according to effectiveness levels, and an iterative item response theory algorithm to obtain estimates of ratees' citizenship performance (W. C. Borman et al, 2001). In the present research, we developed CARS to measure the entire managerial performance domain, including task and citizenship performance, thus addressing a major limitation of the earlier CARS. The paper describes this development effort, including an adjustment to the algorithm that reduces substantially the number of item pairs required to obtain almost as much precision in the performance estimates. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aAlgorithms10aAssociations10aCitizenship10aComputer Assisted Testing10aConstruction10aContextual10aItem Response Theory10aJob Performance10aManagement10aManagement Personnel10aRating Scales10aTest1 aSchneider, RJ1 aGoff, M1 aAnderson, S1 aBorman, W C uhttp://iacat.org/content/computerized-adaptive-rating-scales-measuring-managerial-performance01927nas a2200229 4500008004100000245007200041210006900113300001200182490000700194520116000201653001801361653002101379653003001400653001801430653002501448653002501473653005701498653002201555100001501577700001201592856009301604 2003 eng d00aComputerized adaptive testing using the nearest-neighbors criterion0 aComputerized adaptive testing using the nearestneighbors criteri a204-2160 v273 aItem selection procedures designed for computerized adaptive testing need to accurately estimate every taker's trait level (θ) and, at the same time, effectively use all items in a bank. Empirical studies showed that classical item selection procedures based on maximizing Fisher or other related information yielded highly varied item exposure rates; with these procedures, some items were frequently used whereas others were rarely selected. In the literature, methods have been proposed for controlling exposure rates; they tend to affect the accuracy in θ estimates, however. A modified version of the maximum Fisher information (MFI) criterion, coined the nearest neighbors (NN) criterion, is proposed in this study. The NN procedure improves to a moderate extent the undesirable item exposure rates associated with the MFI criterion and keeps sufficient precision in estimates. The NN criterion will be compared with a few other existing methods in an empirical study using the mean squared errors in θ estimates and plots of item exposure rates associated with different distributions. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10a(Statistical)10aAdaptive Testing10aComputer Assisted Testing10aItem Analysis10aItem Response Theory10aStatistical Analysis10aStatistical Estimation computerized adaptive testing10aStatistical Tests1 aCheng, P E1 aLiou, M uhttp://iacat.org/content/computerized-adaptive-testing-using-nearest-neighbors-criterion01743nas a2200217 4500008004100000245008700041210006900128300001200197490000700209520100200216653002101218653003001239653002601269653002501295653002001320653001401340653004901354100001401403700001401417856009401431 2003 eng d00aItem exposure constraints for testlets in the verbal reasoning section of the MCAT0 aItem exposure constraints for testlets in the verbal reasoning s a335-3560 v273 aThe current study examined item exposure control procedures for testlet scored reading passages in the Verbal Reasoning section of the Medical College Admission Test with four computerized adaptive testing (CAT) systems using the partial credit model. The first system used a traditional CAT using maximum information item selection. The second used random item selection to provide a baseline for optimal exposure rates. The third used a variation of Lunz and Stahl's randomization procedure. The fourth used Luecht and Nungester's computerized adaptive sequential testing (CAST) system. A series of simulated fixed-length CATs was run to determine the optimal item length selection procedure. Results indicated that both the randomization procedure and CAST performed well in terms of exposure control and measurement precision, with the CAST system providing the best overall solution when all variables were taken into consideration. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aEntrance Examinations10aItem Response Theory10aRandom Sampling10aReasoning10aVerbal Ability computerized adaptive testing1 aDavis, LL1 aDodd, B G uhttp://iacat.org/content/item-exposure-constraints-testlets-verbal-reasoning-section-mcat01432nas a2200205 4500008004100000245008800041210007000129300001200199490000700211520067700218653002100895653003000916653002400946653002500970653002600995653005201021100001901073700002301092856011101115 2003 eng d00aOptimal stratification of item pools in α-stratified computerized adaptive testing0 aOptimal stratification of item pools in αstratified computerized a262-2740 v273 aA method based on 0-1 linear programming (LP) is presented to stratify an item pool optimally for use in α-stratified adaptive testing. Because the 0-1 LP model belongs to the subclass of models with a network flow structure, efficient solutions are possible. The method is applied to a previous item pool from the computerized adaptive testing (CAT) version of the Graduate Record Exams (GRE) Quantitative Test. The results indicate that the new method performs well in practical situations. It improves item exposure control, reduces the mean squared error in the θ estimates, and increases test reliability. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Content (Test)10aItem Response Theory10aMathematical Modeling10aTest Construction computerized adaptive testing1 aChang, Hua-Hua1 avan der Linden, WJ uhttp://iacat.org/content/optimal-stratification-item-pools-%CE%B1-stratified-computerized-adaptive-testing01797nas a2200241 4500008004100000245009300041210006900134300001200203490000700215520098200222653001801204653002101222653003001243653001901273653004601292653001801338653002501356653001501381100001401396700001901410700001501429856011101444 2003 eng d00aThe relationship between item exposure and test overlap in computerized adaptive testing0 arelationship between item exposure and test overlap in computeri a129-1450 v403 aThe purpose of this article is to present an analytical derivation for the mathematical form of an average between-test overlap index as a function of the item exposure index, for fixed-length computerized adaptive tests (CATs). This algebraic relationship is used to investigate the simultaneous control of item exposure at both the item and test levels. The results indicate that, in fixed-length CATs, control of the average between-test overlap is achieved via the mean and variance of the item exposure rates of the items that constitute the CAT item pool. The mean of the item exposure rates is easily manipulated. Control over the variance of the item exposure rates can be achieved via the maximum item exposure rate (r-sub(max)). Therefore, item exposure control methods which implement a specification of r-sub(max) (e.g., J. B. Sympson and R. D. Hetter, 1985) provide the most direct control at both the item and test levels. (PsycINFO Database Record (c) 2005 APA )10a(Statistical)10aAdaptive Testing10aComputer Assisted Testing10aHuman Computer10aInteraction computerized adaptive testing10aItem Analysis10aItem Analysis (Test)10aTest Items1 aChen, S-Y1 aAnkenmann, R D1 aSpray, J A uhttp://iacat.org/content/relationship-between-item-exposure-and-test-overlap-computerized-adaptive-testing01545nas a2200193 4500008004100000245026000041210006900301300001000370490000700380520067700387653002101064653002201085653001701107653001501124653004801139100002201187700002201209856012001231 2003 eng d00aTiming behavior in computerized adaptive testing: Response times for correct and incorrect answers are not related to general fluid intelligence/Zum Zeitverhalten beim computergestützten adaptiveb Testen: Antwortlatenzen bei richtigen und falschen Lösun0 aTiming behavior in computerized adaptive testing Response times  a57-630 v243 aExamined the effects of general fluid intelligence on item response times for correct and false responses in computerized adaptive testing. After performing the CFT3 intelligence test, 80 individuals (aged 17-44 yrs) completed perceptual and cognitive discrimination tasks. Results show that response times were related neither to the proficiency dimension reflected by the task nor to the individual level of fluid intelligence. Furthermore, the false > correct-phenomenon as well as substantial positive correlations between item response times for false and correct responses were shown to be independent of intelligence levels. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aCognitive Ability10aIntelligence10aPerception10aReaction Time computerized adaptive testing1 aRammsayer, Thomas1 aBrandler, Susanne uhttp://iacat.org/content/timing-behavior-computerized-adaptive-testing-response-times-correct-and-incorrect-answers02081nas a2200229 4500008004100000245012900041210006900170300001200239490000700251520124000258653001801498653002101516653004501537653003001582653001801612653002501630653002601655100001601681700001401697700001901711856012101730 2002 eng d00aA comparison of item selection techniques and exposure control mechanisms in CATs using the generalized partial credit model0 acomparison of item selection techniques and exposure control mec a147-1630 v263 aThe use of more performance items in large-scale testing has led to an increase in the research investigating the use of polytomously scored items in computer adaptive testing (CAT). Because this research has to be complemented with information pertaining to exposure control, the present research investigated the impact of using five different exposure control algorithms in two sized item pools calibrated using the generalized partial credit model. The results of the simulation study indicated that the a-stratified design, in comparison to a no-exposure control condition, could be used to reduce item exposure and overlap, increase pool utilization, and only minorly degrade measurement precision. Use of the more restrictive exposure control algorithms, such as the Sympson-Hetter and conditional Sympson-Hetter, controlled exposure to a greater extent but at the cost of measurement precision. Because convergence of the exposure control parameters was problematic for some of the more restrictive exposure control algorithms, use of the more simplistic exposure control mechanisms, particularly when the test length to item pool size ratio is large, is recommended. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10a(Statistical)10aAdaptive Testing10aAlgorithms computerized adaptive testing10aComputer Assisted Testing10aItem Analysis10aItem Response Theory10aMathematical Modeling1 aPastor, D A1 aDodd, B G1 aChang, Hua-Hua uhttp://iacat.org/content/comparison-item-selection-techniques-and-exposure-control-mechanisms-cats-using-generalized01443nas a2200241 4500008004100000245008000041210006900121300001200190490000700202520067300209653003000882653002800912653002500940653002300965653001600988653002201004653002101026100001301047700001601060700001001076700001601086856009901102 2002 eng d00aData sparseness and on-line pretest item calibration-scaling methods in CAT0 aData sparseness and online pretest item calibrationscaling metho a207-2180 v393 aCompared and evaluated 3 on-line pretest item calibration-scaling methods (the marginal maximum likelihood estimate with 1 expectation maximization [EM] cycle [OEM] method, the marginal maximum likelihood estimate with multiple EM cycles [MEM] method, and M. L. Stocking's Method B) in terms of item parameter recovery when the item responses to the pretest items in the pool are sparse. Simulations of computerized adaptive tests were used to evaluate the results yielded by the three methods. The MEM method produced the smallest average total error in parameter estimation, and the OEM method yielded the largest total error (PsycINFO Database Record (c) 2005 APA )10aComputer Assisted Testing10aEducational Measurement10aItem Response Theory10aMaximum Likelihood10aMethodology10aScaling (Testing)10aStatistical Data1 aBan, J-C1 aHanson, B A1 aYi, Q1 aHarris, D J uhttp://iacat.org/content/data-sparseness-and-line-pretest-item-calibration-scaling-methods-cat01634nas a2200217 4500008004100000245009700041210006900138300001200207490000700219520087500226653002101101653003001122653002501152653002301177653002501200653002801225653002701253100001301280700001501293856010801308 2002 eng d00aAn EM approach to parameter estimation for the Zinnes and Griggs paired comparison IRT model0 aEM approach to parameter estimation for the Zinnes and Griggs pa a208-2270 v263 aBorman et al. recently proposed a computer adaptive performance appraisal system called CARS II that utilizes paired comparison judgments of behavioral stimuli. To implement this approach,the paired comparison ideal point model developed by Zinnes and Griggs was selected. In this article,the authors describe item response and information functions for the Zinnes and Griggs model and present procedures for estimating stimulus and person parameters. Monte carlo simulations were conducted to assess the accuracy of the parameter estimation procedures. The results indicated that at least 400 ratees (i.e.,ratings) are required to obtain reasonably accurate estimates of the stimulus parameters and their standard errors. In addition,latent trait estimation improves as test length increases. The implications of these results for test construction are also discussed. 10aAdaptive Testing10aComputer Assisted Testing10aItem Response Theory10aMaximum Likelihood10aPersonnel Evaluation10aStatistical Correlation10aStatistical Estimation1 aStark, S1 aDrasgow, F uhttp://iacat.org/content/em-approach-parameter-estimation-zinnes-and-griggs-paired-comparison-irt-model01213nas a2200217 4500008004100000245005100041210005100092300001200143490000700155520060500162653002600767653003000793653001600823653001300839653001300852653001100865653001200876653001500888100001600903856007600919 2002 eng d00aInformation technology and literacy assessment0 aInformation technology and literacy assessment a369-3730 v183 aThis column discusses information technology and literacy assessment in the past and present. The author also describes computer-based assessments today including the following topics: computer-scored testing, computer-administered formal assessment, Internet formal assessment, computerized adaptive tests, placement tests, informal assessment, electronic portfolios, information management, and Internet information dissemination. A model of the major present-day applications of information technologies in reading and literacy assessment is also included. (PsycINFO Database Record (c) 2005 APA )10aComputer Applications10aComputer Assisted Testing10aInformation10aInternet10aLiteracy10aModels10aSystems10aTechnology1 aBalajthy, E uhttp://iacat.org/content/information-technology-and-literacy-assessment01689nas a2200277 4500008004100000020001000041245006500051210006400116260009700180300001100277520074500288653002101033653002201054653002501076653002801101653002501129653001601154653001801170653005501188653001501243653001201258100001801270700002301288700001301311856008701324 2002 eng d a02-0900aMathematical-programming approaches to test item pool design0 aMathematicalprogramming approaches to test item pool design aTwente, The NetherlandsbUniversity of Twente, Faculty of Educational Science and Technology a93-1083 a(From the chapter) This paper presents an approach to item pool design that has the potential to improve on the quality of current item pools in educational and psychological testing and hence to increase both measurement precision and validity. The approach consists of the application of mathematical programming techniques to calculate optimal blueprints for item pools. These blueprints can be used to guide the item-writing process. Three different types of design problems are discussed, namely for item pools for linear tests, item pools computerized adaptive testing (CAT), and systems of rotating item pools for CAT. The paper concludes with an empirical example of the problem of designing a system of rotating item pools for CAT.10aAdaptive Testing10aComputer Assisted10aComputer Programming10aEducational Measurement10aItem Response Theory10aMathematics10aPsychometrics10aStatistical Rotation computerized adaptive testing10aTest Items10aTesting1 aVeldkamp, B P1 avan der Linden, WJ1 aAriel, A uhttp://iacat.org/content/mathematical-programming-approaches-test-item-pool-design01627nas a2200241 4500008004100000245005900041210005800100300001200158490000700170520087100177653002101048653003401069653002801103653002001131653003201151653002501183653001501208653002701223653002201250653001601272100001601288856008101304 2002 eng d00aOutlier detection in high-stakes certification testing0 aOutlier detection in highstakes certification testing a219-2330 v393 aDiscusses recent developments of person-fit analysis in computerized adaptive testing (CAT). Methods from statistical process control are presented that have been proposed to classify an item score pattern as fitting or misfitting the underlying item response theory model in CAT Most person-fit research in CAT is restricted to simulated data. In this study, empirical data from a certification test were used. Alternatives are discussed to generate norms so that bounds can be determined to classify an item score pattern as fitting or misfitting. Using bounds determined from a sample of a high-stakes certification test, the empirical analysis showed that different types of misfit can be distinguished Further applications using statistical process control methods to detect misfitting item score patterns are discussed. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10acomputerized adaptive testing10aEducational Measurement10aGoodness of Fit10aItem Analysis (Statistical)10aItem Response Theory10aperson Fit10aStatistical Estimation10aStatistical Power10aTest Scores1 aMeijer, R R uhttp://iacat.org/content/outlier-detection-high-stakes-certification-testing02030nas a2200253 4500008004100000245010900041210006900150300000900219490000600228520114100234653002101375653001501396653003901411653002201450653002501472653001801497653002201515653005501537653001501592653001201607100001701619700002501636856011501661 2002 eng d00aA structure-based approach to psychological measurement: Matching measurement models to latent structure0 astructurebased approach to psychological measurement Matching me a4-160 v93 aThe present article sets forth the argument that psychological assessment should be based on a construct's latent structure. The authors differentiate dimensional (continuous) and taxonic (categorical) structures at the latent and manifest levels and describe the advantages of matching the assessment approach to the latent structure of a construct. A proper match will decrease measurement error, increase statistical power, clarify statistical relationships, and facilitate the location of an efficient cutting score when applicable. Thus, individuals will be placed along a continuum or assigned to classes more accurately. The authors briefly review the methods by which latent structure can be determined and outline a structure-based approach to assessment that builds on dimensional scaling models, such as item response theory, while incorporating classification methods as appropriate. Finally, the authors empirically demonstrate the utility of their approach and discuss its compatibility with traditional assessment methods and with computerized adaptive testing. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aAssessment10aClassification (Cognitive Process)10aComputer Assisted10aItem Response Theory10aPsychological10aScaling (Testing)10aStatistical Analysis computerized adaptive testing10aTaxonomies10aTesting1 aRuscio, John1 aRuscio, Ayelet Meron uhttp://iacat.org/content/structure-based-approach-psychological-measurement-matching-measurement-models-latent01695nas a2200229 4500008004100000245007800041210006900119300001200188490000700200520094500207653002501152653005101177653003001228653001801258653001101276653002701287653001101314100001701325700001101342700001801353856009401371 2001 eng d00aComputerized adaptive testing with the generalized graded unfolding model0 aComputerized adaptive testing with the generalized graded unfold a177-1960 v253 aExamined the use of the generalized graded unfolding model (GGUM) in computerized adaptive testing. The objective was to minimize the number of items required to produce equiprecise estimates of person locations. Simulations based on real data about college student attitudes toward abortion and on data generated to fit the GGUM were used. It was found that as few as 7 or 8 items were needed to produce accurate and precise person estimates using an expected a posteriori procedure. The number items in the item bank (20, 40, or 60 items) and their distribution on the continuum (uniform locations or item clusters in moderately extreme locations) had only small effects on the accuracy and precision of the estimates. These results suggest that adaptive testing with the GGUM is a good method for achieving estimates with an approximately uniform level of precision using a small number of items. (PsycINFO Database Record (c) 2005 APA )10aAttitude Measurement10aCollege Students computerized adaptive testing10aComputer Assisted Testing10aItem Response10aModels10aStatistical Estimation10aTheory1 aRoberts, J S1 aLin, Y1 aLaughlin, J E uhttp://iacat.org/content/computerized-adaptive-testing-generalized-graded-unfolding-model01965nas a2200193 4500008004100000245008600041210006900127300001000196490000700206520131400213653001401527653003001541653002501571653001101596653003601607653001401643100001501657856009901672 2001 eng d00aDevelopments in measurement of persons and items by means of item response models0 aDevelopments in measurement of persons and items by means of ite a65-940 v283 aThis paper starts with a general introduction into measurement of hypothetical constructs typical of the social and behavioral sciences. After the stages ranging from theory through operationalization and item domain to preliminary test or questionnaire have been treated, the general assumptions of item response theory are discussed. The family of parametric item response models for dichotomous items is introduced and it is explained how parameters for respondents and items are estimated from the scores collected from a sample of respondents who took the test or questionnaire. Next, the family of nonparametric item response models is explained, followed by the 3 classes of item response models for polytomous item scores (e.g., rating scale scores). Then, to what degree the mean item score and the unweighted sum of item scores for persons are useful for measuring items and persons in the context of item response theory is discussed. Methods for fitting parametric and nonparametric models to data are briefly discussed. Finally, the main applications of item response models are discussed, which include equating and item banking, computerized and adaptive testing, research into differential item functioning, person fit research, and cognitive modeling. (PsycINFO Database Record (c) 2005 APA )10aCognitive10aComputer Assisted Testing10aItem Response Theory10aModels10aNonparametric Statistical Tests10aProcesses1 aSijtsma, K uhttp://iacat.org/content/developments-measurement-persons-and-items-means-item-response-models01461nas a2200253 4500008004100000245013900041210006900180260005000249300001200299520053700311653002100848653002500869653001200894653002900906653002200935653002700957653002600984653001501010653001601025100001501041700001601056700001701072856011801089 2001 eng d00aItem response theory applied to combinations of multiple-choice and constructed-response items--approximation methods for scale scores0 aItem response theory applied to combinations of multiplechoice a aMahwah, N.J. USAbLawrence Erlbaum Associates a289-3153 a(From the chapter) The authors develop approximate methods that replace the scoring tables with weighted linear combinations of the component scores. Topics discussed include: a linear approximation for the extension to combinations of scores; the generalization of two or more scores; potential applications of linear approximations to item response theory in computerized adaptive tests; and evaluation of the pattern-of-summed-scores, and Gaussian approximation, estimates of proficiency. (PsycINFO Database Record (c) 2005 APA )10aAdaptive Testing10aItem Response Theory10aMethod)10aMultiple Choice (Testing10aScoring (Testing)10aStatistical Estimation10aStatistical Weighting10aTest Items10aTest Scores1 aThissen, D1 aNelson, L A1 aSwygert, K A uhttp://iacat.org/content/item-response-theory-applied-combinations-multiple-choice-and-constructed-response-items01385nas a2200193 4500008004100000245009400041210006900135300001200204490000700216520067900223653002100902653003000923653002500953653005700978100001401035700001901049700001901068856010401087 2000 eng d00aA comparison of item selection rules at the early stages of computerized adaptive testing0 acomparison of item selection rules at the early stages of comput a241-2550 v243 aThe effects of 5 item selection rules--Fisher information (FI), Fisher interval information (FII), Fisher information with a posterior distribution (FIP), Kullback-Leibler information (KL), and Kullback-Leibler information with a posterior distribution (KLP)--were compared with respect to the efficiency and precision of trait (θ) estimation at the early stages of computerized adaptive testing (CAT). FII, FIP, KL, and KLP performed marginally better than FI at the early stages of CAT for θ=-3 and -2. For tests longer than 10 items, there appeared to be no precision advantage for any of the selection rules. (PsycINFO Database Record (c) 2005 APA ) (journal abstract)10aAdaptive Testing10aComputer Assisted Testing10aItem Analysis (Test)10aStatistical Estimation computerized adaptive testing1 aChen, S-Y1 aAnkenmann, R D1 aChang, Hua-Hua uhttp://iacat.org/content/comparison-item-selection-rules-early-stages-computerized-adaptive-testing00712nas a2200193 4500008004100000245008400041210006900125300001400194490000700208653003000215653001100245653002500256653001600281653005500297653002200352653002300374100001800397856010300415 2000 eng d00aEmergence of item response modeling in instrument development and data analysis0 aEmergence of item response modeling in instrument development an aII60-II650 v3810aComputer Assisted Testing10aHealth10aItem Response Theory10aMeasurement10aStatistical Validity computerized adaptive testing10aTest Construction10aTreatment Outcomes1 aHambleton, RK uhttp://iacat.org/content/emergence-item-response-modeling-instrument-development-and-data-analysis01428nas a2200193 4500008004100000245006300041210006300104300001200167490000700179520079500186653001800981653002100999653003001020653001801050653005701068100001501125700001201140856008201152 2000 eng d00aEstimation of trait level in computerized adaptive testing0 aEstimation of trait level in computerized adaptive testing a257-2650 v243 aNotes that in computerized adaptive testing (CAT), a examinee's trait level (θ) must be estimated with reasonable accuracy based on a small number of item responses. A successful implementation of CAT depends on (1) the accuracy of statistical methods used for estimating θ and (2) the efficiency of the item-selection criterion. Methods of estimating θ suitable for CAT are reviewed, and the differences between Fisher and Kullback-Leibler information criteria for selecting items are discussed. The accuracy of different CAT algorithms was examined in an empirical study. The results show that correcting θ estimates for bias was necessary at earlier stages of CAT, but most CAT algorithms performed equally well for tests of 10 or more items. (PsycINFO Database Record (c) 2005 APA )10a(Statistical)10aAdaptive Testing10aComputer Assisted Testing10aItem Analysis10aStatistical Estimation computerized adaptive testing1 aCheng, P E1 aLiou, M uhttp://iacat.org/content/estimation-trait-level-computerized-adaptive-testing02403nas a2200193 4500008004100000245010700041210006900148300000900217490000700226520169400233653003201927653002501959653002001984653001802004653002602022653001402048100002602062856012102088 2000 eng d00aAn exploratory analysis of item parameters and characteristics that influence item level response time0 aexploratory analysis of item parameters and characteristics that a18120 v613 aThis research examines the relationship between item level response time and (1) item discrimination, (2) item difficulty, (3) word count, (4) item type, and (5) whether a figure is included in an item. Data are from the Graduate Management Admission Test, which is currently offered only as a computerized adaptive test. Analyses revealed significant differences in response time between the five item types: problem solving, data sufficiency, sentence correction, critical reasoning, and reading comprehension. For this reason, the planned pairwise and complex analyses were run within each item type. Pairwise curvilinear regression analyses explored the relationship between response time and item discrimination, item difficulty, and word count. Item difficulty significantly contributed to the prediction of response time for each item type; two of the relationships were significantly quadratic. Item discrimination significantly contributed to the prediction of response time for only two of the item types; one revealed a quadratic relationship and the other a cubic relationship. Word count had significant linear relationship with response time for all the item types except reading comprehension, for which there was no significant relationship. Multiple regression analyses using word count, item difficulty, and item discrimination predicted between 35.4% and 71.4% of the variability in item response time across item types. The results suggest that response time research should consider the type of item that is being administered and continue to explore curvilinear relationships between response time and its predictor variables. (PsycINFO Database Record (c) 2005 APA )10aItem Analysis (Statistical)10aItem Response Theory10aProblem Solving10aReaction Time10aReading Comprehension10aReasoning1 aSmith, Russell Winsor uhttp://iacat.org/content/exploratory-analysis-item-parameters-and-characteristics-influence-item-level-response-time01168nas a2200205 4500008004100000245005600041210005300097300001200150490000700162520055300169653002200722653002500744653002500769653002200794653001500816100002300831700001800854700001500872856007500887 2000 eng d00aAn integer programming approach to item bank design0 ainteger programming approach to item bank design a139-1500 v243 aAn integer programming approach to item bank design is presented that can be used to calculate an optimal blueprint for an item bank, in order to support an existing testing program. The results are optimal in that they minimize the effort involved in producing the items as revealed by current item writing patterns. Also presented is an adaptation of the models, which can be used as a set of monitoring tools in item bank management. The approach is demonstrated empirically for an item bank that was designed for the Law School Admission Test. 10aAptitude Measures10aItem Analysis (Test)10aItem Response Theory10aTest Construction10aTest Items1 avan der Linden, WJ1 aVeldkamp, B P1 aReese, L M uhttp://iacat.org/content/integer-programming-approach-item-bank-design02151nas a2200277 4500008004100000020002200041245009600063210006900159250001500228260000800243300001100251490000700262520122800269653001601497653003901513653003701552653001101589653003301600653002501633653002701658653003101685100001701716700001601733700001701749856010701766 1999 eng d a1040-2446 (Print)00aEvaluating the usefulness of computerized adaptive testing for medical in-course assessment0 aEvaluating the usefulness of computerized adaptive testing for m a1999/10/28 cOct a1125-80 v743 aPURPOSE: This study investigated the feasibility of converting an existing computer-administered, in-course internal medicine test to an adaptive format. METHOD: A 200-item internal medicine extended matching test was used for this research. Parameters were estimated with commercially available software with responses from 621 examinees. A specially developed simulation program was used to retrospectively estimate the efficiency of the computer-adaptive exam format. RESULTS: It was found that the average test length could be shortened by almost half with measurement precision approximately equal to that of the full 200-item paper-and-pencil test. However, computer-adaptive testing with this item bank provided little advantage for examinees at the upper end of the ability continuum. An examination of classical item statistics and IRT item statistics suggested that adding more difficult items might extend the advantage to this group of examinees. CONCLUSIONS: Medical item banks presently used for incourse assessment might be advantageously employed in adaptive testing. However, it is important to evaluate the match between the items and the measurement objective of the test before implementing this format.10a*Automation10a*Education, Medical, Undergraduate10aEducational Measurement/*methods10aHumans10aInternal Medicine/*education10aLikelihood Functions10aPsychometrics/*methods10aReproducibility of Results1 aKreiter, C D1 aFerguson, K1 aGruppen, L D uhttp://iacat.org/content/evaluating-usefulness-computerized-adaptive-testing-medical-course-assessment00557nas a2200121 4500008004100000245011900041210006900160260002100229653003400250653001900284100001400303856011800317 1996 eng d00aA comparison of the traditional maximum information method and the global information method in CAT item selection0 acomparison of the traditional maximum information method and the aNew York, NY USA10acomputerized adaptive testing10aitem selection1 aTang, K L uhttp://iacat.org/content/comparison-traditional-maximum-information-method-and-global-information-method-cat-item