Mixed Integer Linear Programming for Optimizing a Hopfield Network

Abstract

This work presents an approach to optimize the weights of a discrete Hopfield network as mixed integer linear program (MILP). As the original formulation involves a sign-function, it is not differentiable, but parameter optimization using a (mixed integer) LP is possible. As autoassociative memory, a key question is the amount of patterns which can be stored in such a Hopfield network. In this work it is shown, that the traditional storage description models are far inferior to a globally optimized solution which can be obtained with a MILP. In contrast to a gradient descent based optimization is the proposed approach nearly parameter free and independent from seeding and other factors which are crucial for differentiable programming. Additionally it is possible to enforce sparsity constraints on the weights. Such additional constraints improve the generalization of such a model and make the Hopfield network more stable for the case of outliers or missing values. Several experiments demonstrate the effectiveness of the model.