让模糊负样本自动形成学习梯度，提升多模态对齐效果
Aligning by Misaligning: Boundary-aware Curriculum Learning for Multimodal Alignment
- 用边界感知采样器动态筛选难负样本，构建渐进式学习课程
- 对比局部注意力损失定位差异位置，显著提升对齐精度
- 无需额外标注，兼容现有双编码器模型，性能超越CLIP
多数多模态模型将所有负样本同等对待，忽略了仅在细微处与正样本不同的模糊负例。本文提出边界感知课程学习（BACL），一种轻量级可插拔模块，将这些边界案例转化为课程信号。边界感知负样本采样器逐步提高难度，对比局部注意力损失则突出不匹配发生的位置。两个模块完全可微，适用于任何现成的双编码器模型。理论预测误差率快速收敛至O(1/n)；实际表现显示，在四个大规模基准上相较CLIP最高提升+32% R@1，并达到新最优性能，且无需额外标注。
原文摘要 · Abstract (English)
Most multimodal models treat every negative pair alike, ignoring the ambiguous negatives that differ from the positive by only a small detail. We propose Boundary-Aware Curriculum with Local Attention (BACL), a lightweight add-on that turns these borderline cases into a curriculum signal. A Boundary-aware Negative Sampler gradually raises difficulty, while a Contrastive Local Attention loss highlights where the mismatch occurs. The two modules are fully differentiable and work with any off-the-shelf dual encoder. Theory predicts a fast O(1/n) error rate; practice shows up to +32% R@1 over CLIP and new SOTA on four large-scale benchmarks, all without extra labels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。