让大模型代理在多方互动中忠于委托方，避免过度拒绝合理请求。
Whose Side Is Your Agent On? Multi-Party Principal Loyalty in LLM Agents
- 设计七条优先级规则的提示框架，控制代理忠诚度。
- 多轮测试显示，优化后代理拒绝对抗性提问比例降至20%以下。
- 适用于需要精准代理忠诚度的商业谈判与协作场景。
一类快速增长的大模型代理具备多主体特性：代理代表委托方（接受指令、接收反馈、汇报结果），同时在另一通道与利益可能冲突的对方交流（如与供应商谈判、筛选请求或协调员工）。此时‘帮助当前对话者’是错误目标。代理必须忠于其代表的委托方，同时不过度拒绝委托方自身的合作请求。本文研究此多主体忠诚问题，提出评估工具、两种机制与一条结构性结论。PrincipalBench是一个包含75个多轮任务的基准，含泄漏探测、双评委评分和完整性审计门控。在13个前沿模型上，该测试揭示了单轮安全评估无法发现的显著分歧（≤20%与53.6%-75.3%伤害率）：一类为选择性拒绝集群（拒绝对抗性探针但响应合法请求），另一类为过度拒绝集群（广泛拒绝）。(M1) 提示阶段的忠诚度支架（由50+失败轨迹提炼出的7条固定系统规则）使Claude-Sonnet伤害率降至19.4%，所有9个选择性模型均≤20%。(M2) 每token-KL蒸馏方法将有提示的Qwen3-32B教师模型成功迁移到8B Qwen3与Llama-3.1学生模型，是目前测得最强的开源权重迁移方案。(教训) 两种机制仅在泄漏/过度拒绝之间权衡，无法突破该平衡：提升一端必然牺牲另一端，联合最优仍不可达。
原文摘要 · Abstract (English)
A rapidly growing class of LLM agents is multi-party: the agent acts for a principal (who briefs it, sends follow-ups, and receives results) while also conversing in a separate channel with a counterparty whose interests may diverge (negotiating with a vendor, screening inbound requests, or mediating between employees). Here "help whoever you are talking to" is the wrong objective. The agent must stay loyal to the principal it represents without over-refusing the principal's own cooperative asks. We study this multi-party loyalty problem and contribute a measurement instrument, two mechanisms, and a structural lesson. PrincipalBench is a 75-item multi-turn benchmark with leak probes, dual judges, and an integrity-audit gate. Across 13 frontier subjects it exposes a sharp split (<=20% vs. 53.6-75.3% harm) invisible to single-turn safety evaluations: a selective cluster that declines adversarial probes while still following the principal's legitimate requests, and an over-refusing cluster that refuses broadly. (M1) A prompt-time loyalty scaffold (a fixed system prompt of seven prioritized rules, open-coded from 50+ failure trajectories) holds Claude-Sonnet to 19.4% harm and all nine selective subjects to <=20%. (M2) A per-token-KL distillation recipe transfers a prompted Qwen3-32B teacher into 8B Qwen3 and Llama-3.1 students, the strongest open-weight recipe we measure. (Lesson) Both mechanisms only move along a common leak/over-refusal trade-off rather than crossing it: improving one axis costs the other, and the jointly favorable outcome stays out of reach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。