NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms
Abstract
We introduce NotaGen, a symbolic music generation model aiming to explore the potential of producing high-quality classical sheet music. Inspired by the success of Large Language Models (LLMs), NotaGen adopts pre-training, fine-tuning, and reinforcement learning paradigms (henceforth referred to as the LLM training paradigms). It is pre-trained on 1.6M pieces of music in ABC notation, and then fine-tuned on approximately 9K high-quality classical compositions conditioned on "period-composer-instrumentation" prompts. For reinforcement learning, we propose the CLaMP-DPO method, which further enhances generation quality and controllability without requiring human annotations or predefined rewards. Our experiments demonstrate the efficacy of CLaMP-DPO in symbolic music generation models with different architectures and encoding schemes. Furthermore, subjective A/B tests show that NotaGen outperforms baseline models against human compositions, greatly advancing musical aesthetics in symbolic music generation.
Cite
Text
Wang et al. "NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms." International Joint Conference on Artificial Intelligence, 2025. doi:10.24963/IJCAI.2025/1134Markdown
[Wang et al. "NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms." International Joint Conference on Artificial Intelligence, 2025.](https://mlanthology.org/ijcai/2025/wang2025ijcai-notagen/) doi:10.24963/IJCAI.2025/1134BibTeX
@inproceedings{wang2025ijcai-notagen,
title = {{NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms}},
author = {Wang, Yashan and Wu, Shangda and Hu, Jianhuai and Du, Xingjian and Peng, Yueqi and Huang, Yongxin and Fan, Shuai and Li, Xiaobing and Yu, Feng and Sun, Maosong},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2025},
pages = {10207-10215},
doi = {10.24963/IJCAI.2025/1134},
url = {https://mlanthology.org/ijcai/2025/wang2025ijcai-notagen/}
}