用多智能体投票机制消除大模型自评偏见，无需人工标注
Mitigating Judgment Preference Bias in Large Language Models through Group-Based Polling
- 构建多智能体系统模拟投票过程，无监督优化评估模型
- 在多个评测中超越有监督模型，且不依赖人工标注数据
- 对弱模型也有效，适合提升自动评估系统的公平性
大语言模型作为自动评估者（LLM-as-a-Judge）在对齐人类判断方面发挥着关键作用。然而，基于LLM的评估模型常表现出自评偏好，倾向于青睐自身生成的回答，影响评估可靠性。本文提出群体投票优化框架Genii，将多种基于LLM的评估模型集成到多智能体系统中，模拟客户端-服务器交互投票机制，实现无监督优化。实验表明，Genii在无需任何人工标注的情况下，性能优于有监督训练模型；在不同客户端中持续提升表现，即使较弱模型担任服务器也有效。分析显示，Genii显著缓解了评估模型的自评偏好，验证了其有效性。代码已开源。
原文摘要 · Abstract (English)
Large Language Models (LLMs) as automatic evaluators, commonly referred to as LLM-as-a-Judge, have also attracted growing attention. This approach plays a vital role in aligning LLMs with human judgments, providing accurate and reliable assessments. However, LLM-based judgment models often exhibit judgment preference bias during the evaluation phase, tending to favor responses generated by themselves, undermining the reliability of their judgments. This paper introduces the Group-Based Polling Optimization (Genii), an unsupervised multi-agent collaborative optimization framework that mitigates the inherent judgment preference bias of judgment models. Specifically, Genii integrates various LLM-based judgment models into a multi-agent system and simulates the interactive client-server polling mechanism to optimize each client agent unsupervisedly. Our experiments demonstrate that Genii outperforms supervised models trained on annotated judgment data, while requiring no human-labeled annotations. Genii consistently improves performance across different client agents during the polling, even when weaker models act as server agents. Further analysis reveals that Genii effectively mitigates judgment preference bias of LLM-based judgment models, demonstrating its effectiveness. All codes are available at https://github.com/NEUIR/Genii.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。