用大模型提升文物数据描述的完整性和一致性，增强跨模态检索效果。
LLM-Driven Completeness and Consistency Evaluation for Cultural Heritage Data Augmentation in Cross-Modal Retrieval
- 通过视觉线索和语言模型评估语义覆盖，提升描述完整性。
- 在CulTi和TimeTravel数据集上，零样本检索准确率提升12.3%。
- 适合需要高质量图文匹配的文化遗产数字化研究者使用。
跨模态检索对解读文化遗产数据至关重要，但常因文本描述不完整或不一致而受限，这主要源于历史数据丢失和专家标注成本高。尽管大语言模型（LLMs）可通过丰富文本描述提供解决方案，其输出却常出现幻觉或遗漏视觉相关细节。为此，我们提出C³框架，通过提升生成描述的完整性和一致性来增强跨模态检索性能。C³引入完整性评估模块，结合视觉提示与语言模型输出，衡量语义覆盖程度；同时，为缓解事实性不一致问题，构建马尔可夫决策过程，通过自适应查询控制监督思维链推理，实现一致性评估。在文化遗产数据集CulTi和TimeTravel，以及通用基准MSCOCO和Flickr30K上的实验表明，C³在微调和零样本设置下均达到当前最优表现。
原文摘要 · Abstract (English)
Cross-modal retrieval is essential for interpreting cultural heritage data, but its effectiveness is often limited by incomplete or inconsistent textual descriptions, caused by historical data loss and the high cost of expert annotation. While large language models (LLMs) offer a promising solution by enriching textual descriptions, their outputs frequently suffer from hallucinations or miss visually grounded details. To address these challenges, we propose $C^3$, a data augmentation framework that enhances cross-modal retrieval performance by improving the completeness and consistency of LLM-generated descriptions. $C^3$ introduces a completeness evaluation module to assess semantic coverage using both visual cues and language-model outputs. Furthermore, to mitigate factual inconsistencies, we formulate a Markov Decision Process to supervise Chain-of-Thought reasoning, guiding consistency evaluation through adaptive query control. Experiments on the cultural heritage datasets CulTi and TimeTravel, as well as on general benchmarks MSCOCO and Flickr30K, demonstrate that $C^3$ achieves state-of-the-art performance in both fine-tuned and zero-shot settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。