Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks
Abstract
The Boolean satisfiability problem (SAT) is a well-known example of monotonic reasoning, of intense practical interest due to fast solvers, complemented by rigorous fine-grained complexity results. However, for non-monotonic reasoning, e.g., abductive reasoning, comparably little is known outside classic complexity theory. In this paper we take a first step of bridging the gap between monotonic and non-monotonic reasoning by analyzing the complexity of intractable abduction problems under the seemingly overlooked but natural parameter n: the number of variables in the knowledge base. We obtain several positive results for SigmaP2- as well as NP- and coNP-complete fragments, which implies the first example of beating exhaustive search for a SigmaP2-complete problem (to the best of our knowledge). We complement this with lower bounds and for many fragments rule out improvements under the (strong) exponential-time hypothesis.
Cite
Text
Liu et al. "Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks." International Joint Conference on Artificial Intelligence, 2024. doi:10.24963/ijcai.2024/508Markdown
[Liu et al. "Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks." International Joint Conference on Artificial Intelligence, 2024.](https://mlanthology.org/ijcai/2024/liu2024ijcai-provable/) doi:10.24963/ijcai.2024/508BibTeX
@inproceedings{liu2024ijcai-provable,
title = {{Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks}},
author = {Liu, Xin and Tao, Wei and Li, Wei and Zhan, Dazhi and Wang, Jun and Pan, Zhisong},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2024},
pages = {4596-4604},
doi = {10.24963/ijcai.2024/508},
url = {https://mlanthology.org/ijcai/2024/liu2024ijcai-provable/}
}