用柯西-施瓦茨散度提升跨模态检索的稳定性和效果
Cross-Modal Retrieval with Cauchy-Schwarz Divergence
- 提出无超参的柯西-施瓦茨散度，增强训练稳定性
- 在六大数据集上实现比现有方法更优的跨模态检索性能
- 可统一处理双模与三模数据，避免繁琐的两两对比
有效的跨模态检索需要对异构数据类型进行稳健对齐。现有方法多聚焦于双模任务，依赖诸如KL散度、最大均值差异和相关性对齐等分布对齐技术，但常面临数值不稳定、对超参数敏感以及无法捕捉分布全结构等问题。本文提出柯西-施瓦茨（CS）散度，一种无超参的度量，显著提升训练稳定性和检索性能。进一步基于霍尔德不等式提出广义CS（GCS）散度，通过双向循环比较机制，在统一数学框架下直接实现三模及以上模态对齐，无需穷举两两比较。在六个基准数据集上的大量实验表明，该方法在双模与三模检索任务中均表现优异。代码已公开于https://github.com/JiahaoZhang666/CSD。
原文摘要 · Abstract (English)
Effective cross-modal retrieval requires robust alignment of heterogeneous data types. Most existing methods focus on bi-modal retrieval tasks and rely on distributional alignment techniques such as Kullback-Leibler divergence, Maximum Mean Discrepancy, and correlation alignment. However, these methods often suffer from critical limitations, including numerical instability, sensitivity to hyperparameters, and their inability to capture the full structure of the underlying distributions. In this paper, we introduce the Cauchy-Schwarz (CS) divergence, a hyperparameter-free measure that improves both training stability and retrieval performance. We further propose a novel Generalized CS (GCS) divergence inspired by Hölder's inequality. This extension enables direct alignment of three or more modalities within a unified mathematical framework through a bidirectional circular comparison scheme, eliminating the need for exhaustive pairwise comparisons. Extensive experiments on six benchmark datasets demonstrate the effectiveness of our method in both bi-modal and tri-modal retrieval tasks. The code of our CS/GCS divergence is publicly available at https://github.com/JiahaoZhang666/CSD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。