让大模型代理的决策过程可审计，追踪每项结论的证据链。
LEDGER: Claim-to-Evidence Trace Graphs for Auditing LLM Agents
- 构建分层证据图，把执行记录聚类为证据与工作流节点。
- 用语义边连接主张与支持动作、产物和验证步骤，显式展现推理路径。
- 适合需要可信审查的复杂任务，如科研或工程自动化场景。
大语言模型（LLM）代理现在能执行涉及复杂工具调用、代码执行、文件编辑和生成产物的长周期技术流程。随着代理效率提升，生产瓶颈从输出转向审核输出是否正确可信。现有的可观测系统虽能展示细粒度执行事件，但仅靠可见性仍需审查者自行重建哪些操作、产物和验证步骤对特定结论至关重要。我们提出LEDGER——用于执行审查的分层证据与决策图，通过在观测到的代理会话上构建分层追溯图，保留原始追踪记录的同时，将它们归类为证据节点和工作流节点，以产物作为证据锚点，并添加类型化的语义边，连接主张与其支持的动作、产物和检查。通过数据分析与编码示例，我们展示了这些追踪结果如何揭示工作流决策、产物溯源、修复步骤、验证覆盖范围及以证据为中心的主张支持路径。
原文摘要 · Abstract (English)
Large language model (LLM) agents can now carry out long-horizon technical workflows involving complex tool use, code execution, file edits, and generated artifacts. As agents do more work faster, the productivity bottleneck shifts from producing outputs to auditing whether those outputs are correct and trustworthy. Agent observability systems make fine-grained execution events visible, but visibility alone still leaves reviewers to reconstruct which actions, artifacts, and validation steps matter for a particular conclusion. We introduce LEDGER - Layered Evidence and Decision Graphs for Execution Review, a tracing and review system that builds layered trace graphs over observed agent sessions. LEDGER preserves Trace Records while grouping them into Evidence Nodes and Workflow Nodes, representing artifacts as evidence anchors, and adding typed semantic edges that connect claims to supporting actions, artifacts, and checks. Through data-analysis and coding examples, we show how the resulting traces expose workflow decisions, artifact lineage, repair steps, validation coverage, and claim-support paths for evidence-centered audit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。