研究大模型在法律摘要中如何因角色不同而选择性呈现信息。
Modeling Motivated Reasoning in Law: Evaluating Strategic Role Conditioning in LLM Summarization
- 通过不同法律角色提示，测试大模型摘要时的信息选择策略。
- 即使有平衡指令，模型仍偏向角色立场，出现选择性信息呈现。
- 提醒法律场景下需关注模型角色感知风险，适合法律AI评估者参考。
大型语言模型（LLMs）被越来越多地用于生成针对特定利益相关者的摘要，适应不同用户需求。在法律领域，这引发了一个重要问题：模型是否会表现出动机性推理——即为迎合法律体系中特定利益方的立场而战略性地呈现信息？基于法律现实主义理论和当前法律实践趋势，我们研究了当要求模型以不同法律角色（如法官、检察官、律师）身份总结司法判决时，其响应模式。我们提出一个基于法律事实与推理内容包含率的评估框架，并考虑对利益相关方的倾向性。结果显示，即便提示中包含平衡要求，模型仍表现出反映角色一致性的选择性信息包含模式。这一发现警示，当大模型开始从过往交互或上下文中推断用户角色时，类似对齐行为可能自发产生，即使无明确角色指令。研究强调，在高风险法律场景中，必须开展角色敏感的模型摘要行为评估。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly used to generate user-tailored summaries, adapting outputs to specific stakeholders. In legal contexts, this raises important questions about motivated reasoning -- how models strategically frame information to align with a stakeholder's position within the legal system. Building on theories of legal realism and recent trends in legal practice, we investigate how LLMs respond to prompts conditioned on different legal roles (e.g., judges, prosecutors, attorneys) when summarizing judicial decisions. We introduce an evaluation framework grounded in legal fact and reasoning inclusion, also considering favorability towards stakeholders. Our results show that even when prompts include balancing instructions, models exhibit selective inclusion patterns that reflect role-consistent perspectives. These findings raise broader concerns about how similar alignment may emerge as LLMs begin to infer user roles from prior interactions or context, even without explicit role instructions. Our results underscore the need for role-aware evaluation of LLM summarization behavior in high-stakes legal settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。