Using Knowledge to Speed Learning: A Comparison of Knowledge-Based Cascade-Correlation and Multi-Task Learning

Abstract

Cognitive modeling with neural networks unrealistically ignores the role of knowledge in learning by starting from random weights. It is likely that effective use of knowledge by neural networks could significantly speed learning. A new algorithm, knowledge-based cascadecorrelation (KBCC), finds and adapts its relevant knowledge in new learning. Comparison to multi-task learning (MTL) reveals that KBCC uses its knowledge more effectively to learn faster. 1. Existing Knowledge and New Learning Neural networks typically learn de novo without the benefit of existing knowledge. However, when people learn, they routinely use their knowledge (Pazzani, 1991; Wisniewski, 1995). Such use of prior knowledge in learning is likely responsible for the ease and speed with which people learn, and for interference with new learning. The technical reason that neural networks fail to use knowledge is that they begin learning from initially random connection weights. This implements a tabula rasa view of each distinct learning task that very few cognitive psychologists would accept. In this paper, we compare two algorithms (KBCC and MTL) for their ability to use knowledge to speed learning. KBCC is an extension of cascade-correlation (CC), a generative learning algorithm often used in the simulation of cognitive development (Buckingham & Shultz, in