Adaptive Gradient Descent on Riemannian Manifolds and Its Applications to Gaussian Variational Inference
Abstract
We propose RAdaGD, a novel family of adaptive gradient descent methods on general Riemannian manifolds. RAdaGD adapts the step size parameter without line search, and includes instances that achieve a non-ergodic convergence guarantee, $f(x_k) - f(x_\star) \le \mathcal{O}(1/k)$, under local geodesic smoothness and generalized geodesic convexity. A core application of RAdaGD is Gaussian Variational Inference, where our method provides the first convergence guarantee in the absence of $L$-smoothness of the target log-density, under additional technical assumptions. We also investigate the empirical performance of RAdaGD in numerical simulations and demonstrate its competitiveness in comparison to existing algorithms.
Cite
Text
Park et al. "Adaptive Gradient Descent on Riemannian Manifolds and Its Applications to Gaussian Variational Inference." International Conference on Learning Representations, 2026.Markdown
[Park et al. "Adaptive Gradient Descent on Riemannian Manifolds and Its Applications to Gaussian Variational Inference." International Conference on Learning Representations, 2026.](https://mlanthology.org/iclr/2026/park2026iclr-adaptive/)BibTeX
@inproceedings{park2026iclr-adaptive,
title = {{Adaptive Gradient Descent on Riemannian Manifolds and Its Applications to Gaussian Variational Inference}},
author = {Park, Jiyoung and Suh, Jaewook J. and Wang, Bofan and Bhattacharya, Anirban and Ma, Shiqian},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mlanthology.org/iclr/2026/park2026iclr-adaptive/}
}