Distributed Learning of Non-Convex Linear Models with One Round of Communication
Abstract
We present the optimal weighted average (OWA) distributed learning algorithm for linear models. OWA achieves statistically optimal learning rates, uses only one round of communication, works on non-convex problems, and supports a fast cross validation procedure. The OWA algorithm first trains local models on each of the compute nodes; then a master machine merges the models using a second round of optimization. This second optimization uses only a small fraction of the data, and so has negligible computational cost. Compared with similar distributed estimators that merge locally trained models, OWA either has stronger statistical guarantees, is applicable to more models, or has a more computationally efficient merging procedure.
Cite
Text
Izbicki and Shelton. "Distributed Learning of Non-Convex Linear Models with One Round of Communication." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2019. doi:10.1007/978-3-030-46147-8_12Markdown
[Izbicki and Shelton. "Distributed Learning of Non-Convex Linear Models with One Round of Communication." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2019.](https://mlanthology.org/ecmlpkdd/2019/izbicki2019ecmlpkdd-distributed/) doi:10.1007/978-3-030-46147-8_12BibTeX
@inproceedings{izbicki2019ecmlpkdd-distributed,
title = {{Distributed Learning of Non-Convex Linear Models with One Round of Communication}},
author = {Izbicki, Mike and Shelton, Christian R.},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2019},
pages = {197-212},
doi = {10.1007/978-3-030-46147-8_12},
url = {https://mlanthology.org/ecmlpkdd/2019/izbicki2019ecmlpkdd-distributed/}
}