CAT is an exception to the traditional conception of validity. It is one of the few examples of individualized testing. Item difficulty is tailored to each examinee. The intent, however, is increased efficiency. Focus on reliability (reduced standard error); Equivalence with paper & pencil tests is valued; Validity is enhanced through improved reliability.

How Else Might We Individualize Testing Using CAT?

• By addressing construct-irrelevant factors influencing individual test scores (usually in negatively biased ways).
• Individual Score Validity (ISV) – how free is a particular score from construct-irrelevant factors (often called construct-irrelevant variance, or CIV).

An ISV-Based View of Validity

Test Event -- An examinee encounters a series of items in a particular context.

• •All 3 elements (examinee, items, context) are potential sources of CIV.
  • Examples:
    • Test anxiety (examinee)
    • Amount/difficulty of reading required (item)
    • Test stakes (context)
• ISV can be affected by all 3 elements.

CAT Goal: individualize testing to address CIV threats to score validity (i.e., maximize ISV).

Some Research Issues:

• What are some innovative methods for expanding CAT that address ISV threats while preserving measurement of the target construct?
• How might CAT help address the ISV challenges posed by test anxiety?
• How should policy-makers deal with scores that have been shown to have low ISV?