01362nas a2200145 4500008003900000022001400039245005800053210005800111300001400169490000700183520095400190100001901144700001201163856004101175 2016 d a1745-398400aMonitoring Items in Real Time to Enhance CAT Security0 aMonitoring Items in Real Time to Enhance CAT Security a131–1510 v533 aAn IRT-based sequential procedure is developed to monitor items for enhancing test security. The procedure uses a series of statistical hypothesis tests to examine whether the statistical characteristics of each item under inspection have changed significantly during CAT administration. This procedure is compared with a previously developed CTT-based procedure through simulation studies. The results show that when the total number of examinees is fixed both procedures can control the rate of type I errors at any reasonable significance level by choosing an appropriate cutoff point and meanwhile maintain a low rate of type II errors. Further, the IRT-based method has a much lower type II error rate or more power than the CTT-based method when the number of compromised items is small (e.g., 5), which can be achieved if the IRT-based procedure can be applied in an active mode in the sense that flagged items can be replaced with new items.1 aZhang, Jinming1 aLi, Jie uhttp://dx.doi.org/10.1111/jedm.1210401176nas a2200121 4500008003900000245009200039210006900131300001100200490000700211520076500218100001900983856005201002 2014 d00aA Sequential Procedure for Detecting Compromised Items in the Item Pool of a CAT System0 aSequential Procedure for Detecting Compromised Items in the Item a87-1040 v383 a
To maintain the validity of a continuous testing system, such as computerized adaptive testing (CAT), items should be monitored to ensure that the performance of test items has not gone through any significant changes during their lifetime in an item pool. In this article, the author developed a sequentially monitoring procedure based on a series of statistical hypothesis tests to examine whether the statistical characteristics of individual items have changed significantly during test administration. Simulation studies show that under the simulated setting, by choosing an appropriate cutoff point, the procedure can control the rate of Type I errors at any reasonable significance level and meanwhile, has a very low rate of Type II errors.
1 aZhang, Jinming uhttp://apm.sagepub.com/content/38/2/87.abstract