用强化学习让图像描述自动修正错误，提升生成准确性。
SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning
- 通过场景图解析对比初始与修正后的描述，设计奖励函数
- 在RefinedCaps数据集上，显著优于直接偏好优化方法
- 适合需要高精度图像描述的视觉语言任务研究者
我们提出SC-Captioner，一种基于强化学习的框架，使图像描述模型具备自我修正能力。关键在于设计奖励函数，激励准确修正。通过场景图解析算法将预测与参考描述分解为物体、属性和关系集合，计算初始与修正后集合的差异，匹配参考集合以获得正确修正的奖励和错误增删的惩罚，形成最终奖励信号。针对描述质量评估，我们改进了CAPTURE指标，解决其不完整精确率评估和低效关系匹配问题。此外，我们构建了一个细粒度标注的图像描述数据集RefinedCaps，包含6.5K来自COCO的多样化图像。实验表明，将SC-Captioner应用于大型视觉-语言模型，可在多种场景下生成更优图像描述，显著超越直接偏好优化训练策略。
原文摘要 · Abstract (English)
We propose SC-Captioner, a reinforcement learning framework that enables the self-correcting capability of image caption models. Our crucial technique lies in the design of the reward function to incentivize accurate caption corrections. Specifically, the predicted and reference captions are decomposed into object, attribute, and relation sets using scene-graph parsing algorithms. We calculate the set difference between sets of initial and self-corrected captions to identify added and removed elements. These elements are matched against the reference sets to calculate correctness bonuses for accurate refinements and mistake punishments for wrong additions and removals, thereby forming the final reward. For image caption quality assessment, we propose a set of metrics refined from CAPTURE that alleviate its incomplete precision evaluation and inefficient relation matching problems. Furthermore, we collect a fine-grained annotated image caption dataset, RefinedCaps, consisting of 6.5K diverse images from COCO dataset. Experiments show that applying SC-Captioner on large visual-language models can generate better image captions across various scenarios, significantly outperforming the direct preference optimization training strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。