用反思笔记提升图像描述的准确与细节，让大模型不瞎编也不漏关键信息。
ReflectCAP: Detailed Image Captioning with Reflective Memory
- 通过多智能体分析大模型常犯错和忽略之处，生成可复用的指导笔记
- 在8个主流视觉语言模型上实现事实性与细节覆盖的最优平衡
- 比模型扩容或传统多智能体方案节省21%-36%算力，适合实际部署
详细图像描述需要兼顾事实准确性与细粒度覆盖，但现有方法难以同时实现。本文提出反射笔记引导的图像描述（ReflectCAP），通过多智能体流水线分析目标大视觉语言模型（LVLM）反复出现的幻觉和系统性遗漏，提炼出可复用的结构化反思笔记。推理时，这些笔记同时指引模型规避错误与关注关键信息，显著提升描述的准确性和完整性。该方法应用于涵盖GPT-4.1系列、Qwen系列及InternVL变体的8个LVLM，达到事实性与覆盖度之间的帕累托最优，并在CapArena-Auto评测中显著优于强参考模型。此外，相比模型扩容或现有多智能体方案，ReflectCAP仅增加21%-36%计算开销，实现高质量描述与成本效率的更好平衡，使高精度图像描述在真实场景中更具可行性。
原文摘要 · Abstract (English)
Detailed image captioning demands both factual grounding and fine-grained coverage, yet existing methods have struggled to achieve them simultaneously. We address this tension with Reflective Note-Guided Captioning (ReflectCAP), where a multi-agent pipeline analyzes what the target large vision-language model (LVLM) consistently hallucinates and what it systematically overlooks, distilling these patterns into reusable guidelines called Structured Reflection Notes. At inference time, these notes steer the captioning model along both axes -- what to avoid and what to attend to -- yielding detailed captions that jointly improve factuality and coverage. Applying this method to 8 LVLMs spanning the GPT-4.1 family, Qwen series, and InternVL variants, ReflectCAP reaches the Pareto frontier of the trade-off between factuality and coverage, and delivers substantial gains on CapArena-Auto, where generated captions are judged head-to-head against strong reference models. Moreover, ReflectCAP offers a more favorable trade-off between caption quality and compute cost than model scaling or existing multi-agent pipelines, which incur 21--36\% greater overhead. This makes high-quality detailed captioning viable under real-world cost and latency constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。