A Deep Reinforcement Learning Framework for Optimal Trade Execution
Abstract
In this article, we propose a deep reinforcement learning based framework to learn to minimize trade execution costs by splitting a sell order into child orders and execute them sequentially over a fixed period. The framework is based on a variant of the Deep Q-Network (DQN) algorithm that integrates the Double DQN, Dueling Network, and Noisy Nets. In contrast to previous research work, which uses implementation shortfall as the immediate rewards, we use a shaped reward structure, and we also incorporate the zero-ending inventory constraint into the DQN algorithm by slightly modifying the Q-function updates relative to standard Q-learning at the final step. We demonstrate that the DQN based optimal trade execution framework (1) converges fast during the training phase, (2) outperforms TWAP, VWAP, AC and 2 DQN algorithms during the backtesting on 14 US equities, and also (3) improves the stability by incorporating the zero ending inventory constraint.
Cite
Text
Lin and Beling. "A Deep Reinforcement Learning Framework for Optimal Trade Execution." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020. doi:10.1007/978-3-030-67670-4_14Markdown
[Lin and Beling. "A Deep Reinforcement Learning Framework for Optimal Trade Execution." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020.](https://mlanthology.org/ecmlpkdd/2020/lin2020ecmlpkdd-deep/) doi:10.1007/978-3-030-67670-4_14BibTeX
@inproceedings{lin2020ecmlpkdd-deep,
title = {{A Deep Reinforcement Learning Framework for Optimal Trade Execution}},
author = {Lin, Siyu and Beling, Peter A.},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2020},
pages = {223-240},
doi = {10.1007/978-3-030-67670-4_14},
url = {https://mlanthology.org/ecmlpkdd/2020/lin2020ecmlpkdd-deep/}
}