Lurie Networks with K-Contracting Dynamics

Abstract

This paper proposes an approach to enable the weights and biases of a novel neural ODE, the Lurie network, to be trained in such a manner that a generalised concept of stability is guaranteed. This generalised stability measure is derived through the use of k-contraction analysis, which guarantees global convergence to a point, line or plane in the neural state-space. An unconstrained parametrisation of this condition is derived, allowing models to be trained using standard optimisation algorithms, whilst limiting the search space to solutions satisfying the k-contraction constraint. The novel stability result and parametrisation provide a toolset for training over the space of Lurie network's which exhibit the convergent behaviours observed during neural computation in the brain. For example, global convergence to one of multiple equilibrium points or limit cycles are properties observed in associative and working memory.