A Multiscale Attentional Framework for Relaxation Neural Networks
Abstract
We investigate the optimization of neural networks governed by general objective functions. Practical formulations of such objec(cid:173) tives are notoriously difficult to solve; a common problem is the poor local extrema that result by any of the applied methods. In this paper, a novel framework is introduced for the solution oflarge(cid:173) scale optimization problems. It assumes little about the objective function and can be applied to general nonlinear, non-convex func(cid:173) tions; objectives in thousand of variables are thus efficiently min(cid:173) imized by a combination of techniques - deterministic annealing, multiscale optimization, attention mechanisms and trust region op(cid:173) timization methods.
Cite
Text
Tsioutsias and Mjolsness. "A Multiscale Attentional Framework for Relaxation Neural Networks." Neural Information Processing Systems, 1995.Markdown
[Tsioutsias and Mjolsness. "A Multiscale Attentional Framework for Relaxation Neural Networks." Neural Information Processing Systems, 1995.](https://mlanthology.org/neurips/1995/tsioutsias1995neurips-multiscale/)BibTeX
@inproceedings{tsioutsias1995neurips-multiscale,
title = {{A Multiscale Attentional Framework for Relaxation Neural Networks}},
author = {Tsioutsias, Dimitris I. and Mjolsness, Eric},
booktitle = {Neural Information Processing Systems},
year = {1995},
pages = {633-639},
url = {https://mlanthology.org/neurips/1995/tsioutsias1995neurips-multiscale/}
}