Video Motion Transfer with Diffusion Transformers
Abstract
We propose DiTFlow, a method for transferring the motion of a reference video to a newly synthesized one, designed specifically for Diffusion Transformers (DiT). We first process the reference video with a pre-trained DiT to analyze cross-frame attention maps and extract a patch-wise motion signal called the Attention Motion Flow (AMF). We guide the latent denoising process in an optimization-based, training-free, manner by optimizing latents with our AMF loss to generate videos reproducing the motion of the reference one. We also apply our optimization strategy to transformer positional embeddings, granting us a boost in zero-shot motion transfer capabilities. We evaluate DiTFlow against recently published methods, outperforming all across multiple metrics and human evaluation.
Cite
Text
Pondaven et al. "Video Motion Transfer with Diffusion Transformers." Conference on Computer Vision and Pattern Recognition, 2025. doi:10.1109/CVPR52734.2025.02133Markdown
[Pondaven et al. "Video Motion Transfer with Diffusion Transformers." Conference on Computer Vision and Pattern Recognition, 2025.](https://mlanthology.org/cvpr/2025/pondaven2025cvpr-video/) doi:10.1109/CVPR52734.2025.02133BibTeX
@inproceedings{pondaven2025cvpr-video,
title = {{Video Motion Transfer with Diffusion Transformers}},
author = {Pondaven, Alexander and Siarohin, Aliaksandr and Tulyakov, Sergey and Torr, Philip and Pizzati, Fabio},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2025},
pages = {22911-22921},
doi = {10.1109/CVPR52734.2025.02133},
url = {https://mlanthology.org/cvpr/2025/pondaven2025cvpr-video/}
}