PyTorch: An Imperative Style, High-Performance Deep Learning Library
Abstract
Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it was designed from first principles to support an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several commonly used benchmarks.
Cite
Text
Paszke et al. "PyTorch: An Imperative Style, High-Performance Deep Learning Library." Neural Information Processing Systems, 2019.Markdown
[Paszke et al. "PyTorch: An Imperative Style, High-Performance Deep Learning Library." Neural Information Processing Systems, 2019.](https://mlanthology.org/neurips/2019/paszke2019neurips-pytorch/)BibTeX
@inproceedings{paszke2019neurips-pytorch,
title = {{PyTorch: An Imperative Style, High-Performance Deep Learning Library}},
author = {Paszke, Adam and Gross, Sam and Massa, Francisco and Lerer, Adam and Bradbury, James and Chanan, Gregory and Killeen, Trevor and Lin, Zeming and Gimelshein, Natalia and Antiga, Luca and Desmaison, Alban and Kopf, Andreas and Yang, Edward and DeVito, Zachary and Raison, Martin and Tejani, Alykhan and Chilamkurthy, Sasank and Steiner, Benoit and Fang, Lu and Bai, Junjie and Chintala, Soumith},
booktitle = {Neural Information Processing Systems},
year = {2019},
pages = {8026-8037},
url = {https://mlanthology.org/neurips/2019/paszke2019neurips-pytorch/}
}