ML Anthology

Structured Packing in LLM Training Improves Long Context Utilization

Konrad Staniszewski, Szymon Tworkowski, Sebastian Jaszczur, Yu Zhao, Henryk Michalewski, Lukasz Kucinski, Piotr Milos
AAAI 2025 pp. 25201-25209
doi:10.1609/AAAI.V39I24.34706 /aaai/2025/staniszewski2025aaai-structured/
AAAI Semantic Scholar

Cite

Text

Staniszewski et al. "Structured Packing in LLM Training Improves Long Context Utilization." AAAI Conference on Artificial Intelligence, 2025. doi:10.1609/AAAI.V39I24.34706

Markdown

[Staniszewski et al. "Structured Packing in LLM Training Improves Long Context Utilization." AAAI Conference on Artificial Intelligence, 2025.](https://mlanthology.org/aaai/2025/staniszewski2025aaai-structured/) doi:10.1609/AAAI.V39I24.34706

BibTeX

@inproceedings{staniszewski2025aaai-structured,
  title     = {{Structured Packing in LLM Training Improves Long Context Utilization}},
  author    = {Staniszewski, Konrad and Tworkowski, Szymon and Jaszczur, Sebastian and Zhao, Yu and Michalewski, Henryk and Kucinski, Lukasz and Milos, Piotr},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year      = {2025},
  pages     = {25201-25209},
  doi       = {10.1609/AAAI.V39I24.34706},
  url       = {https://mlanthology.org/aaai/2025/staniszewski2025aaai-structured/}
}