提出一种新评分方法，能公平评估多个智能体的策略表现。
Deviation Ratings: A General, Clone-Invariant Rating Method
- 基于粗相关均衡构建通用评分机制
- 支持多人非零和博弈场景且不受重复策略干扰
- 适用于大模型评估等复杂多智能体场景
许多现实中的多智能体或多任务评估场景可自然建模为正规形式博弈，因其固有的战略互动（对抗、合作或混合动机）。这些互动可能是代理性的（如玩家力求获胜）、根本性的（如成本与质量权衡）或互补性的（如细分定位与专业化）。在此框架下，被评分的是策略（动作、策略、智能体、模型、任务、提示等）。然而，多智能体战略交互的冗余性和复杂性使评分变得困难：重复或相似的策略会扭曲其对应策略的评分。先前工作提出了“克隆不变”评分以处理此类冗余，但仅限于两人零和（即严格竞争）互动。本文首次提出基于粗相关均衡的N人一般和克隆不变评分方法，称为偏差评分（Deviation Ratings），并在多个领域进行了探索，包括大语言模型评估。
原文摘要 · Abstract (English)
Many real-world multi-agent or multi-task evaluation scenarios can be naturally modelled as normal-form games due to inherent strategic (adversarial, cooperative, and mixed motive) interactions. These strategic interactions may be agentic (e.g. players trying to win), fundamental (e.g. cost vs quality), or complementary (e.g. niche finding and specialization). In such a formulation, it is the strategies (actions, policies, agents, models, tasks, prompts, etc.) that are rated. However, the rating problem is complicated by redundancy and complexity of N-player strategic interactions. Repeated or similar strategies can distort ratings for those that counter or complement them. Previous work proposed ``clone invariant'' ratings to handle such redundancies, but this was limited to two-player zero-sum (i.e. strictly competitive) interactions. This work introduces the first N-player general-sum clone invariant rating, called deviation ratings, based on coarse correlated equilibria. The rating is explored on several domains including LLMs evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。