Software Testing by Active Learning for Commercial Games

Abstract

As software systems have become larger, exhaustive testing has become increasingly onerous. This has rendered statistical software testing and machine learning techniques increasingly attractive. Drawing from both of these, we present an active learning framework for blackbox software testing. The active learning approach samples input/output pairs from a blackbox and learns a model of the system’s behaviour. This model is then used to select new inputs for sampling. This framework has been developed in the context of commercial video games, complex virtual worlds with highdimensional state spaces, too large for exhaustive testing. Beyond its correctness, developers need to evaluate the gameplay of a game, properties such as difculty . We use the learned model not only to guide sampling but also to summarize the game’s behaviour for the developer to evaluate. We present results from our semi-automated gameplay analysis by machine learning (SAGA-ML) tool applied to Electronics Arts’ FIFA Soccer game.