An Assessment of Submissions Made to the Predictive Toxicology Evaluation Challenge

Abstract

Constructing "good " models for chemical carcinogenesis was identified in IJCAI-97 as providing a substantial challenge to "knowledge discovery " programs. Attention was drawn to a comparative exercise which called for predictions on the outcome of 30 rodent carcinogenicity bioassays. This- the Predictive Toxicology Evaluation (or PTE) Challenge- was seen to provide AI programs with an opportunity to participate in an enterprise of scientific merit, and a yardstick for comparison against strong competition. Here we provide an assessment of the machine learning (ML) submissions made. Models submitted are assessed on: (1) their accuracy, in comparison to models developed with expert collaboration; and (2) their explanatory value for toxicology. The principal findings were: (a) using structural information available from a standard modelling package, layman-devised features, and outcomes of established biological tests, results from MLderived models were at least as good as those with expert-derived techniques. This was surprising; (b) the combined use of structural and biological features by ML-derived models was unusual, and suggested new avenues for toxicology modelling. This was also unexpected; and (c) significant effort was required to interpret the output of even the most "symbolic " of ML-derived models. Much of this could have been alleviated with measures for converting the results into a more "toxicology-friendly" form. As it stands, their absence is sufficient to prevent a whole-hearted acceptance of these promising methods by toxicologists. This suggests that ML techniques have been able to respond- not fully, but nevertheless substantially- to the PTE Challenge.