FutureFill: Fast Generation from Convolutional Sequence Models
Abstract
We address the challenge of efficient auto-regressive generation in sequence prediction models by introducing FutureFill—a general-purpose fast generation method for any sequence prediction algorithm based on convolutional operators. FutureFill reduces generation time from quadratic to quasilinear in the context length. Moreover, when generating from a prompt, it requires a prefill cache whose size grows only with the number of tokens to be generated—often much smaller than the caches required by standard convolutional or attention‐based models. We validate our theoretical claims with language modeling experiments and demonstrate substantial efficiency gains when generating from a deep convolutional sequence prediction model.
Cite
Text
Agarwal et al. "FutureFill: Fast Generation from Convolutional Sequence Models." International Conference on Learning Representations, 2026.Markdown
[Agarwal et al. "FutureFill: Fast Generation from Convolutional Sequence Models." International Conference on Learning Representations, 2026.](https://mlanthology.org/iclr/2026/agarwal2026iclr-futurefill/)BibTeX
@inproceedings{agarwal2026iclr-futurefill,
title = {{FutureFill: Fast Generation from Convolutional Sequence Models}},
author = {Agarwal, Naman and Chen, Xinyi and Dogariu, Evan and Shah, Devan and Strauss, Hubert and Feinberg, Vladimir and Suo, Daniel and Bartlett, Peter and Hazan, Elad},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mlanthology.org/iclr/2026/agarwal2026iclr-futurefill/}
}