GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration
Abstract
Online batch selection methods offer an adaptive alternative to static training data selection by dynamically selecting data batches during training. However, existing methods either rely on impractical reference models or simple heuristics that may not capture true data informativeness. To address these limitations, we propose \emph{GREedy Approximation Taylor Selection} (GREATS), a principled and efficient online batch selection method that applies greedy algorithm to optimize the data batch quality approximated by Taylor expansion. We develop a series of techniques to scale GREATS to large-scale model training. Extensive experiments with large language models (LLMs) demonstrate that GREATS significantly improves training convergence speed and generalization performance.
Cite
Text
Wang et al. "GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration." Neural Information Processing Systems, 2024. doi:10.52202/079017-4169Markdown
[Wang et al. "GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration." Neural Information Processing Systems, 2024.](https://mlanthology.org/neurips/2024/wang2024neurips-greats/) doi:10.52202/079017-4169BibTeX
@inproceedings{wang2024neurips-greats,
title = {{GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration}},
author = {Wang, Jiachen T. and Wu, Tong and Song, Dawn and Mittal, Prateek and Jia, Ruoxi},
booktitle = {Neural Information Processing Systems},
year = {2024},
doi = {10.52202/079017-4169},
url = {https://mlanthology.org/neurips/2024/wang2024neurips-greats/}
}