Machine Life-Long Learning with csMTL Networks

Abstract

Multiple task learning (MTL) neural networks are one of the better documented methods of inductive transfer of task knowledge (Caruana 1997). An MTL network is a feed-forward multi-layer network with an output node for each task being learned. The standard back-propagation of error learning algorithm is used to train all tasks in parallel. The sharing of internal representation in the hidden nodes is the method by which inductive bias occurs between related tasks within an MTL network (Baxter 1996). Previously, (Silver & Mercer 2002; Silver & Poirier 2004) have investigated the use of MTL networks as a basis for de-veloping machine lifelong learning (ML3) systems and have found them to have several limitations caused by the multi-ple outputs. In response to these problems, this article in-troduces context-sensitive MTL, or csMTL, and describes a