Graphical Model Based Computer Adaptive Testing

Abstract

This paper synthesizes ideas from the fields of graphical modelling and eductational testing, particularly Item Response Theory (IRT) applied to Computerized Adaptive Testing (CAT). Graphical modelling can offer IRT a language for describing multifaceted skills and knowledge and disentangling evidence from com- plex performances. IRT-CA T can offer graphical modellers several ways of treating sources of variability other than including more variables in the model. In particular, variables can enter into the modelling pro- cess at several levels: (1) in validity studies (but not in the ordinarily used model), (2) in task construction (in particular, in defining link parameters), (3) in test or model assembly (blocking and randomization con- straints in selecting tasks or other model pieces), (4) in response characterization (i.e. as part of task models which characterize a response) or (5) in the main (student) model. The paper describes an implementation of these ideas in a fielded application: HYDRIVE, a tutor for hydraulics diagnosis