Inductive Transfer with Context-Sensitive Neural Networks

Abstract

Context-sensitive Multiple Task Learning, or cs MTL, is presented as a method of inductive transfer which uses a single output neural network and additional contextual inputs for learning multiple tasks. Motivated by problems with the application of MTL networks to machine lifelong learning systems, cs MTL encoding of multiple task examples was developed and found to improve predictive performance. As evidence, the cs MTL method is tested on seven task domains and shown to produce hypotheses for primary tasks that are often better than standard MTL hypotheses when learning in the presence of related and unrelated tasks. We argue that the reason for this performance improvement is a reduction in the number of effective free parameters in the cs MTL network brought about by the shared output node and weight update constraints due to the context inputs. An examination of IDT and SVM models developed from cs MTL encoded data provides initial evidence that this improvement is not shared across all machine learning models.