PredRNN++: Towards a Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
Abstract
We present PredRNN++, a recurrent network for spatiotemporal predictive learning. In pursuit of a great modeling capability for short-term video dynamics, we make our network deeper in time by leveraging a new recurrent structure named Causal LSTM with cascaded dual memories. To alleviate the gradient propagation difficulties in deep predictive models, we propose a Gradient Highway Unit, which provides alternative quick routes for the gradient flows from outputs back to long-range previous inputs. The gradient highway units work seamlessly with the causal LSTMs, enabling our model to capture the short-term and the long-term video dependencies adaptively. Our model achieves state-of-the-art prediction results on both synthetic and real video datasets, showing its power in modeling entangled motions.
Cite
Text
Wang et al. "PredRNN++: Towards a Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning." International Conference on Machine Learning, 2018.Markdown
[Wang et al. "PredRNN++: Towards a Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning." International Conference on Machine Learning, 2018.](https://mlanthology.org/icml/2018/wang2018icml-predrnn/)BibTeX
@inproceedings{wang2018icml-predrnn,
title = {{PredRNN++: Towards a Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning}},
author = {Wang, Yunbo and Gao, Zhifeng and Long, Mingsheng and Wang, Jianmin and Yu, Philip S},
booktitle = {International Conference on Machine Learning},
year = {2018},
pages = {5123-5132},
volume = {80},
url = {https://mlanthology.org/icml/2018/wang2018icml-predrnn/}
}