Learning with Discrete Multi-Valued Neurons

Abstract

Analog neural networks of limited precision are essentially k-ary neural networks. That is, their processors classify the input space into k regions using k − 1 parallel hyperplanes by computing k-ary weighted multilinear threshold functions. The ability of k-ary neural networks to learn k-ary weighted multilinear threshold functions is examined. The well known perceptron learning algorithm is generalized to a k-ary perceptron algorithm with guaranteed convergence property. Littlestone's winnow algorithm is superior to the perceptron learning algorithm when the ratio of the sum of the weights to the threshold value of the function being learned is small. A k-ary winnow algorithm with a mistake bound which depends on this value and the ratio between the largest and smallest thresholds is presented.