Correlated Quantization for Distributed Mean Estimation and Optimization
Abstract
We study the problem of distributed mean estimation and optimization under communication constraints. We propose a correlated quantization protocol whose error guarantee depends on the deviation of data points instead of their absolute range. The design doesn’t need any prior knowledge on the concentration property of the dataset, which is required to get such dependence in previous works. We show that applying the proposed protocol as a sub-routine in distributed optimization algorithms leads to better convergence rates. We also prove the optimality of our protocol under mild assumptions. Experimental results show that our proposed algorithm outperforms existing mean estimation protocols on a diverse set of tasks.
Cite
Text
Suresh et al. "Correlated Quantization for Distributed Mean Estimation and Optimization." International Conference on Machine Learning, 2022.Markdown
[Suresh et al. "Correlated Quantization for Distributed Mean Estimation and Optimization." International Conference on Machine Learning, 2022.](https://mlanthology.org/icml/2022/suresh2022icml-correlated/)BibTeX
@inproceedings{suresh2022icml-correlated,
title = {{Correlated Quantization for Distributed Mean Estimation and Optimization}},
author = {Suresh, Ananda Theertha and Sun, Ziteng and Ro, Jae and Yu, Felix},
booktitle = {International Conference on Machine Learning},
year = {2022},
pages = {20856-20876},
volume = {162},
url = {https://mlanthology.org/icml/2022/suresh2022icml-correlated/}
}