Local Steps Speed up Local GD for Heterogeneous Distributed Logistic Regression
Abstract
We analyze two variants of Local Gradient Descent applied to distributed logistic regression with heterogeneous, separable data and show convergence at the rate $O(1/KR)$ for $K$ local steps and sufficiently large $R$ communication rounds. In contrast, all existing convergence guarantees for Local GD applied to any problem are at least $\Omega(1/R)$, meaning they fail to show the benefit of local updates. The key to our improved guarantee is showing progress on the logistic regression objective when using a large stepsize $\eta \gg 1/K$, whereas prior analysis depends on $\eta \leq 1/K$.
Cite
Text
Crawshaw et al. "Local Steps Speed up Local GD for Heterogeneous Distributed Logistic Regression." International Conference on Learning Representations, 2025.Markdown
[Crawshaw et al. "Local Steps Speed up Local GD for Heterogeneous Distributed Logistic Regression." International Conference on Learning Representations, 2025.](https://mlanthology.org/iclr/2025/crawshaw2025iclr-local/)BibTeX
@inproceedings{crawshaw2025iclr-local,
title = {{Local Steps Speed up Local GD for Heterogeneous Distributed Logistic Regression}},
author = {Crawshaw, Michael and Woodworth, Blake and Liu, Mingrui},
booktitle = {International Conference on Learning Representations},
year = {2025},
url = {https://mlanthology.org/iclr/2025/crawshaw2025iclr-local/}
}