MLlib: Machine Learning in Apache Spark
Abstract
Apache Spark is a popular open-source platform for large-scale data processing that is well-suited for iterative machine learning tasks. In this paper we present MLlib, Spark's open- source distributed machine learning library. MLlib provides efficient functionality for a wide range of learning settings and includes several underlying statistical, optimization, and linear algebra primitives. Shipped with Spark, MLlib supports several languages and provides a high-level API that leverages Spark's rich ecosystem to simplify the development of end-to-end machine learning pipelines. MLlib has experienced a rapid growth due to its vibrant open-source community of over 140 contributors, and includes extensive documentation to support further growth and to let users quickly get up to speed.
Cite
Text
Meng et al. "MLlib: Machine Learning in Apache Spark." Machine Learning Open Source Software, 2016.Markdown
[Meng et al. "MLlib: Machine Learning in Apache Spark." Machine Learning Open Source Software, 2016.](https://mlanthology.org/mloss/2016/meng2016jmlr-mllib/)BibTeX
@article{meng2016jmlr-mllib,
title = {{MLlib: Machine Learning in Apache Spark}},
author = {Meng, Xiangrui and Bradley, Joseph and Yavuz, Burak and Sparks, Evan and Venkataraman, Shivaram and Liu, Davies and Freeman, Jeremy and Tsai, Db and Amde, Manish and Owen, Sean and Xin, Doris and Xin, Reynold and Franklin, Michael J. and Zadeh, Reza and Zaharia, Matei and Talwalkar, Ameet},
journal = {Machine Learning Open Source Software},
year = {2016},
pages = {1-7},
volume = {17},
url = {https://mlanthology.org/mloss/2016/meng2016jmlr-mllib/}
}