Refined Analysis of Constant Step Size Federated Averaging and Federated Richardson-Romberg Extrapolation
Abstract
In this paper, we present a novel analysis of $\texttt{FedAvg}$ with constant step size, relying on the Markov property of the underlying process. We demonstrate that the global iterates of the algorithm converge to a stationary distribution and analyze its resulting bias and variance relative to the problem’s solution. We provide a first-order bias expansion in both homogeneous and heterogeneous settings. Interestingly, this bias decomposes into two distinct components: one that depends solely on stochastic gradient noise and another on client heterogeneity. Finally, we introduce a new algorithm based on the Richardson-Romberg extrapolation technique to mitigate this bias.
Cite
Text
Mangold et al. "Refined Analysis of Constant Step Size Federated Averaging and Federated Richardson-Romberg Extrapolation." Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, 2025.Markdown
[Mangold et al. "Refined Analysis of Constant Step Size Federated Averaging and Federated Richardson-Romberg Extrapolation." Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, 2025.](https://mlanthology.org/aistats/2025/mangold2025aistats-refined/)BibTeX
@inproceedings{mangold2025aistats-refined,
title = {{Refined Analysis of Constant Step Size Federated Averaging and Federated Richardson-Romberg Extrapolation}},
author = {Mangold, Paul and Durmus, Alain Oliviero and Dieuleveut, Aymeric and Samsonov, Sergey and Moulines, Eric},
booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics},
year = {2025},
pages = {5023-5031},
volume = {258},
url = {https://mlanthology.org/aistats/2025/mangold2025aistats-refined/}
}