Learning to Generalize: An Information Perspective on Neural Processes
Abstract
Neural Processes (NPs) combine the adaptability of neural networks with the efficiency of meta-learning, offering a powerful framework for modeling stochastic processes. However, existing methods focus on empirical performance while lacking a rigorous theoretical understanding of generalization. To address this, we propose an information-theoretic framework to analyze the generalization bounds of NPs, introducing dynamical stability regularization to minimize sharpness and improve optimization dynamics. Additionally, we show how noise-injected parameter updates complement this regularization. The proposed approach, applicable to a wide range of NP models, is validated through experiments on classic benchmarks, including 1D regression, image completion, Bayesian optimization, and contextual bandits. The results demonstrate tighter generalization bounds and superior predictive performance, establishing a principled foundation for advancing generalizable NP models.
Cite
Text
Li et al. "Learning to Generalize: An Information Perspective on Neural Processes." Advances in Neural Information Processing Systems, 2025.Markdown
[Li et al. "Learning to Generalize: An Information Perspective on Neural Processes." Advances in Neural Information Processing Systems, 2025.](https://mlanthology.org/neurips/2025/li2025neurips-learning-a/)BibTeX
@inproceedings{li2025neurips-learning-a,
title = {{Learning to Generalize: An Information Perspective on Neural Processes}},
author = {Li, Hui and Liu, Huafeng and Lin, Shuyang and Shi, Jingyue and Fu, Yiran and Jing, Liping},
booktitle = {Advances in Neural Information Processing Systems},
year = {2025},
url = {https://mlanthology.org/neurips/2025/li2025neurips-learning-a/}
}