Aligning by Misaligning: Boundary-Aware Curriculum Learning for Multimodal Alignment
Abstract
Most multimodal models treat every negative pair alike, ignoring the ambiguous negatives that differ from the positive by only a small detail. We propose Boundary-A ware Curriculum with Local Attention(BACL), a lightweight add-on that turns these borderline cases into a curriculum signal. A Boundary-aware Negative Sampler gradually raises difficulty, while a Contrastive Local Attention loss highlights where the mismatch occurs. The two modules are fully differentiable and work with any off-the-shelf dual encoder. Theory predicts a fast $\tilde{\mathcal{O}}(1/n)$ error rate; practice shows up to +32 \% R@1 over CLIP and new SOTA on four large-scale benchmarks, all without extra labels.
Cite
Text
Ye et al. "Aligning by Misaligning: Boundary-Aware Curriculum Learning for Multimodal Alignment." Advances in Neural Information Processing Systems, 2025.Markdown
[Ye et al. "Aligning by Misaligning: Boundary-Aware Curriculum Learning for Multimodal Alignment." Advances in Neural Information Processing Systems, 2025.](https://mlanthology.org/neurips/2025/ye2025neurips-aligning/)BibTeX
@inproceedings{ye2025neurips-aligning,
title = {{Aligning by Misaligning: Boundary-Aware Curriculum Learning for Multimodal Alignment}},
author = {Ye, Hua and Ding, Hang and Chen, Siyuan and Jiang, Yiyang and Changyuan, Zhang and Zhang, Xuan},
booktitle = {Advances in Neural Information Processing Systems},
year = {2025},
url = {https://mlanthology.org/neurips/2025/ye2025neurips-aligning/}
}