VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling
Abstract
In this work, we systematically study music generation conditioned solely on the video. First, we present a large-scale dataset by collecting 360K video-music pairs, including various genres such as movie trailers, advertisements, and documentaries. Furthermore, we propose VidMuse, a simple framework for generating music aligned with video inputs. VidMuse stands out by producing high-fidelity music that is both acoustically and semantically aligned with the video. By incorporating local and global visual cues, VidMuse enables the creation of coherent music tracks that consistently match the video content through Long-Short-Term modeling. Through extensive experiments, VidMuse outperforms existing models in terms of audio quality, diversity, and audio-visual alignment. The code and datasets are available at https://vidmuse.github.io/
Cite
Text
Tian et al. "VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling." Conference on Computer Vision and Pattern Recognition, 2025. doi:10.1109/CVPR52734.2025.01750Markdown
[Tian et al. "VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling." Conference on Computer Vision and Pattern Recognition, 2025.](https://mlanthology.org/cvpr/2025/tian2025cvpr-vidmuse/) doi:10.1109/CVPR52734.2025.01750BibTeX
@inproceedings{tian2025cvpr-vidmuse,
title = {{VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling}},
author = {Tian, Zeyue and Liu, Zhaoyang and Yuan, Ruibin and Pan, Jiahao and Liu, Qifeng and Tan, Xu and Chen, Qifeng and Xue, Wei and Guo, Yike},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2025},
pages = {18782-18793},
doi = {10.1109/CVPR52734.2025.01750},
url = {https://mlanthology.org/cvpr/2025/tian2025cvpr-vidmuse/}
}