Cyclades: Conflict-Free Asynchronous Machine Learning
Abstract
We present Cyclades, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. Cyclades is asynchronous during model updates, and requires no memory locking mechanisms, similar to Hogwild!-type algorithms. Unlike Hogwild!, Cyclades introduces no conflicts during parallel execution, and offers a black-box analysis for provable speedups across a large family of algorithms. Due to its inherent cache locality and conflict-free nature, our multi-core implementation of Cyclades consistently outperforms Hogwild!-type algorithms on sufficiently sparse datasets, leading to up to 40% speedup gains compared to Hogwild!, and up to 5\times gains over asynchronous implementations of variance reduction algorithms.
Cite
Text
Pan et al. "Cyclades: Conflict-Free Asynchronous Machine Learning." Neural Information Processing Systems, 2016.Markdown
[Pan et al. "Cyclades: Conflict-Free Asynchronous Machine Learning." Neural Information Processing Systems, 2016.](https://mlanthology.org/neurips/2016/pan2016neurips-cyclades/)BibTeX
@inproceedings{pan2016neurips-cyclades,
title = {{Cyclades: Conflict-Free Asynchronous Machine Learning}},
author = {Pan, Xinghao and Lam, Maximilian and Tu, Stephen and Papailiopoulos, Dimitris and Zhang, Ce and Jordan, Michael I and Ramchandran, Kannan and Ré, Christopher},
booktitle = {Neural Information Processing Systems},
year = {2016},
pages = {2568-2576},
url = {https://mlanthology.org/neurips/2016/pan2016neurips-cyclades/}
}