用模糊奖励提升视觉语言模型的群体计数精度。
CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward
- 引入模糊分组相对策略奖励，替代传统0/1准确率奖励。
- 在5个域内数据集上超越GPT4o、LLaMA2等基线模型。
- 适合对答案精度要求高的任务，如精确数量预测。
我们提出一种新型框架FGRPR，将分组相对策略优化（GRPO）与模糊奖励函数结合，以提升学习效率。不同于传统的0/1准确率奖励，该模糊奖励模型提供更细致的激励，促进输出更精确。实验表明，使用标准0/1奖励的GRPO表现低于监督微调（SFT）。而采用FGRPR的Qwen2.5-VL(3B和7B)模型在五个域内数据集上全面超越所有基线，包括GPT4o、LLaMA2(90B)和SFT。在域外数据集上，其性能接近SFT，但在目标值较大时表现更优，因其模糊奖励函数对更接近真实值的预测给予更高奖励。该方法适用于对答案精度要求高的任务。代码与数据：https://github.com/yeyimilk/CrowdVLM-R1
原文摘要 · Abstract (English)
We propose Fuzzy Group Relative Policy Reward (FGRPR), a novel framework that integrates Group Relative Policy Optimization (GRPO) with a fuzzy reward function to enhance learning efficiency. Unlike the conventional binary 0/1 accuracy reward, our fuzzy reward model provides nuanced incentives, encouraging more precise outputs. Experimental results demonstrate that GRPO with a standard 0/1 accuracy reward underperforms compared to supervised fine-tuning (SFT). In contrast, FGRPR, applied to Qwen2.5-VL(3B and 7B), surpasses all baseline models, including GPT4o, LLaMA2(90B), and SFT, across five in-domain datasets. On an out-of-domain dataset, FGRPR achieves performance comparable to SFT but excels when target values are larger, as its fuzzy reward function assigns higher rewards to closer approximations. This approach is broadly applicable to tasks where the precision of the answer is critical. Code and data: https://github.com/yeyimilk/CrowdVLM-R1
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。