TY - JOUR T1 - Automated Simultaneous Assembly of Multistage Testlets for a High-Stakes Licensing Examination JF - Educational and Psychological Measurement Y1 - 2007 A1 - Breithaupt, Krista A1 - Hare, Donovan R. AB -

Many challenges exist for high-stakes testing programs offering continuous computerized administration. The automated assembly of test questions to exactly meet content and other requirements, provide uniformity, and control item exposure can be modeled and solved by mixed-integer programming (MIP) methods. A case study of the computerized licensing examination of the American Institute of Certified Public Accountants is offered as one application of MIP techniques for test assembly. The solution illustrates assembly for a computer-adaptive multistage testing design. However, the general form of the constraint-based solution can be modified to generate optimal test designs for paper-based or computerized administrations, regardless of the specific psychometric model. An extension of this methodology allows for long-term planning for the production and use of test content on the basis of an exact psychometric test designs and administration schedules.

VL - 67 UR - http://epm.sagepub.com/content/67/1/5.abstract ER - TY - JOUR T1 - Detecting Differential Speededness in Multistage Testing JF - Journal of Educational Measurement Y1 - 2007 A1 - van der Linden, Wim J. A1 - Breithaupt, Krista A1 - Chuah, Siang Chee A1 - Zhang, Yanwei AB -

A potential undesirable effect of multistage testing is differential speededness, which happens if some of the test takers run out of time because they receive subtests with items that are more time intensive than others. This article shows how a probabilistic response-time model can be used for estimating differences in time intensities and speed between subtests and test takers and detecting differential speededness. An empirical data set for a multistage test in the computerized CPA Exam was used to demonstrate the procedures. Although the more difficult subtests appeared to have items that were more time intensive than the easier subtests, an analysis of the residual response times did not reveal any significant differential speededness because the time limit appeared to be appropriate. In a separate analysis, within each of the subtests, we found minor but consistent patterns of residual times that are believed to be due to a warm-up effect, that is, use of more time on the initial items than they actually need.

VL - 44 UR - http://dx.doi.org/10.1111/j.1745-3984.2007.00030.x ER - TY - JOUR T1 - Optimal Testlet Pool Assembly for Multistage Testing Designs JF - Applied Psychological Measurement Y1 - 2006 A1 - Ariel, Adelaide A1 - Veldkamp, Bernard P. A1 - Breithaupt, Krista AB -

Computerized multistage testing (MST) designs require sets of test questions (testlets) to be assembled to meet strict, often competing criteria. Rules that govern testlet assembly may dictate the number of questions on a particular subject or may describe desirable statistical properties for the test, such as measurement precision. In an MST design, testlets of differing difficulty levels must be created. Statistical properties for assembly of the testlets can be expressed using item response theory (IRT) parameters. The testlet test information function (TIF) value can be maximized at a specific point on the IRT ability scale. In practical MST designs, parallel versions of testlets are needed, so sets of testlets with equivalent properties are built according to equivalent specifications. In this project, the authors study the use of a mathematical programming technique to simultaneously assemble testlets to ensure equivalence and fairness to candidates who may be administered different testlets.

VL - 30 UR - http://apm.sagepub.com/content/30/3/204.abstract ER - TY - JOUR T1 - A testlet assembly design for the uniform CPA Examination JF - Applied Measurement in Education Y1 - 2006 A1 - Luecht, Richard A1 - Brumfield, Terry A1 - Breithaupt, Krista VL - 19 UR - http://www.tandfonline.com/doi/abs/10.1207/s15324818ame1903_2 ER - TY - JOUR T1 - Automated Simultaneous Assembly for Multistage Testing JF - International Journal of Testing Y1 - 2005 A1 - Breithaupt, Krista A1 - Ariel, Adelaide A1 - Veldkamp, Bernard P. VL - 5 UR - http://www.tandfonline.com/doi/abs/10.1207/s15327574ijt0503_8 ER -