用外部文章增强图像描述，让生成的句子更懂时间地点人物。
CIAN: Multi-Stage Framework for Event-Enriched Image Captioning via Retrieval-Augmented Generation
- 分三阶段：查文章、摘要引导、语法优化
- 在OpenEvents-V1上CIDEr提升至0.094，检索mAP达0.979
- 适合需要背景信息的图像描述任务
事件丰富型图像描述不仅包含可见内容，还涵盖事件的时间、地点和参与者等上下文信息，这是大多数基于像素的模型所缺失的能力。我们提出上下文图像-文章叙述器（CIAN），一个通过检索增强生成的多阶段框架，以补充外部叙事。CIAN使用SigLIP检索相关文章，通过摘要引导微调后的Qwen模型进行叙事生成，并采用基于N-Gram的精细化处理提升语句流畅性与连贯性。在OpenEvents-V1基准测试中，CIAN实现高检索性能（mAP 0.979），并显著提升描述质量，使CIDEr得分从0.030提升至0.094。结果表明，检索增强推理结合语言精炼在生成具有上下文感知能力、类人化描述方面具有显著有效性。
原文摘要 · Abstract (English)
Event-enriched image captioning describes not only visible content but also the broader context of events, including timing, location, and participants, capabilities missing in most pixel-bound models. We propose the Contextual Image-Article Narrator (CIAN), a multi-stage framework that enriches captions with external narratives. CIAN retrieves relevant articles using SigLIP, summarizes them to guide a Narrative Generation stage with a LoRA-fine-tuned Qwen model, and applies N-Gram-based Refinement for fluency and coherence. On the OpenEvents-V1 benchmark, CIAN achieves high retrieval performance (mAP 0.979) and improves caption quality, increasing CIDEr from 0.030 to 0.094. These results highlight the effectiveness of retrieval-augmented reasoning combined with linguistic refinement for generating context-aware, human-like captions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。