提升长图文描述评估精度，兼顾效率与人类判断一致性。
SPECS: Specificity-Enhanced CLIP-Score for Long Image Caption Evaluation
- 基于CLIP改进目标，强化细节准确性奖励与错误惩罚
- 与开源LLM评估方法相关性相当，但计算成本更低
- 适合迭代训练时快速评估长图文生成模型
随着对生成长篇详细图文描述的兴趣增长，传统评估指标可靠性下降。基于n-gram的指标虽高效，却难以捕捉语义正确性；表示相似性（RS）类指标曾因计算开销大而应用受限，即便硬件进步，仍因与人类判断相关性低而未普及。基于大语言模型（LLM）的指标虽与人类判断高度相关，但代价过高，不适于模型开发中的反复评估。本文提出SPECS（Specificity-Enhanced CLIPScore），一种针对长图文描述设计的无参考RS指标。SPECS通过新目标函数改进CLIP，强调细节特异性：奖励正确细节，惩罚错误信息。实验表明，SPECS在与人类判断的相关性上达到开源LLM指标水平，同时效率显著更高，适用于图像描述模型开发中的迭代检查点评估。代码已公开于https://github.com/mbzuai-nlp/SPECS。
原文摘要 · Abstract (English)
As interest grows in generating long, detailed image captions, standard evaluation metrics become increasingly unreliable. N-gram-based metrics though efficient, fail to capture semantic correctness. Representational Similarity (RS) metrics, designed to address this, initially saw limited use due to high computational costs, while today, despite advances in hardware, they remain unpopular due to low correlation to human judgments. Meanwhile, metrics based on large language models (LLMs) show strong correlation with human judgments, but remain too expensive for iterative use during model development. We introduce SPECS (Specificity-Enhanced CLIPScore), a reference-free RS metric tailored to long image captioning. SPECS modifies CLIP with a new objective that emphasizes specificity: rewarding correct details and penalizing incorrect ones. We show that SPECS matches the performance of open-source LLM-based metrics in correlation to human judgments, while being far more efficient. This makes it a practical alternative for iterative checkpoint evaluation during image captioning model development.Our code can be found at https://github.com/mbzuai-nlp/SPECS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。