Dense Associative Memory with Epanechnikov Energy

Abstract

We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum-exponential (LSE) function, LSR is based on the Epanechnikov kernel and enables exact memory retrieval with exponential capacity without requiring exponential separation functions. Uniquely, it introduces abundant additional emergent local minima while preserving perfect pattern recovery--a characteristic previously unseen in DenseAM literature. Empirical results show LSR generates significantly more local minima and produces samples with higher log-likelihood than LSE-based models, making it promising for both memory storage and generative tasks.