用角色化辩论模拟透明决策,提升高风险表格任务的可解释性。
AgenticSimLaw: A Juvenile Courtroom Multi-Agent Debate Simulation for Explainable High-Stakes Tabular Decision Making
- 设定检方、辩护方、法官角色,7轮结构化辩论生成推理过程。
- 在NLSY97数据集上表现更稳定,准确率与F1得分相关性更强。
- 适合需要透明度和人工监管的司法、医疗等高风险决策场景。
我们提出AgenticSimLaw,一种基于角色的多智能体辩论框架,用于高风险表格决策任务的可解释、可控制的运行时推理。不同于黑箱方法,该法庭式架构明确定义了代理角色(检方、辩护方、法官)、交互协议(7轮结构化辩论)和私有推理策略,实现完整的可审计决策流程。我们在使用NLSY97数据集的青年成人再犯预测任务上进行基准测试,对比了近90种模型与策略组合下的传统链式思考(CoT)提示方法。结果表明,结构化多智能体辩论相比单智能体推理具有更稳定的泛化性能,且准确率与F1分数相关性更强。除性能提升外,AgenticSimLaw还提供对推理步骤的细粒度控制,生成完整交互记录以支持可解释性,并支持对代理行为的系统性分析。尽管我们将其应用于刑事司法领域以检验伦理复杂情境下的推理能力,该方法可推广至任何需要透明性和人类监督的协商式高风险决策任务。本工作解决了基于大模型的多智能体系统关键挑战:通过结构化角色实现组织性,通过日志交互实现可观测性,通过显式非部署约束实现责任归属。数据、结果与代码将开源于github.com,采用MIT许可。
原文摘要 · Abstract (English)
We introduce AgenticSimLaw, a role-structured, multi-agent debate framework that provides transparent and controllable test-time reasoning for high-stakes tabular decision-making tasks. Unlike black-box approaches, our courtroom-style orchestration explicitly defines agent roles (prosecutor, defense, judge), interaction protocols (7-turn structured debate), and private reasoning strategies, creating a fully auditable decision-making process. We benchmark this framework on young adult recidivism prediction using the NLSY97 dataset, comparing it against traditional chain-of-thought (CoT) prompting across almost 90 unique combinations of models and strategies. Our results demonstrate that structured multi-agent debate provides more stable and generalizable performance compared to single-agent reasoning, with stronger correlation between accuracy and F1-score metrics. Beyond performance improvements, AgenticSimLaw offers fine-grained control over reasoning steps, generates complete interaction transcripts for explainability, and enables systematic profiling of agent behaviors. While we instantiate this framework in the criminal justice domain to stress-test reasoning under ethical complexity, the approach generalizes to any deliberative, high-stakes decision task requiring transparency and human oversight. This work addresses key LLM-based multi-agent system challenges: organization through structured roles, observability through logged interactions, and responsibility through explicit non-deployment constraints for sensitive domains. Data, results, and code will be available on github.com under the MIT license.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。