On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks
Abstract
This paper empirically studies commonly observed training difficulties of Physics-Informed Neural Networks (PINNs) on dynamical systems. Our results indicate that fixed points which are inherent to these systems play a key role in the optimization of the in PINNs embedded physics loss function. We observe that the loss landscape exhibits local optima that are shaped by the presence of fixed points. We find that these local optima contribute to the complexity of the physics loss optimization which can explain common training difficulties and resulting nonphysical predictions. Under certain settings, e.g., initial conditions close to fixed points or long simulations times, we show that those optima can even become better than that of the desired solution.
Cite
Text
Rohrhofer et al. "On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks." Transactions on Machine Learning Research, 2023.Markdown
[Rohrhofer et al. "On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks." Transactions on Machine Learning Research, 2023.](https://mlanthology.org/tmlr/2023/rohrhofer2023tmlr-role/)BibTeX
@article{rohrhofer2023tmlr-role,
title = {{On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks}},
author = {Rohrhofer, Franz M. and Posch, Stefan and Gößnitzer, Clemens and Geiger, Bernhard C},
journal = {Transactions on Machine Learning Research},
year = {2023},
url = {https://mlanthology.org/tmlr/2023/rohrhofer2023tmlr-role/}
}