构建病理学自动化工作流评估基准,验证AI能否可靠完成临床级病理分析。
Evaluating Agentic Harness Systems for Autonomous Computational Pathology

- 基于病理任务设计可复用的评估框架,适配AI代理系统
- 测试9个模型在41项任务中生成369条完整工作流轨迹
- 发现工具执行与反思修正能力薄弱,临床边界合规性需加强
自主计算病理学(ACP)将高层次病理分析目标转化为可执行、可追溯且符合临床约束的工作流。实现这一能力需将通用智能体框架适配至病理学特定任务、工具、证据标准和临床判断边界。本文提出ACP-Bench,一个从病理辅助系统向完整ACP工作流能力迁移的评估框架。该基准涵盖41项病理工作流任务,包括24项生物标志物、7项形态学和10项预后任务,覆盖6个器官系统和9类终点。评估9个模型及3类框架(Claude Code、Codex、Open Code),生成369条完整轨迹。通过专家审核流程、诊断评估和病理科医生验证的安全审查,综合评估每条轨迹的工作流执行、诊断性能与临床边界一致性。结果显示,任务启动、意图理解与报告生成相对成熟,而工具调用、结果整合与自我修正能力仍不足,端到端完整闭环极为罕见。ACP-Bench为评估智能体系统在临床自主前是否真正具备病理工作流操作能力提供了可复用的标准。
原文摘要 · Abstract (English)
Autonomous computational pathology (ACP) converts high-level pathology analysis goals into executable, traceable and clinically bounded workflows. Realizing this capability requires adapting general agentic harness systems to pathology-specific tasks, tools, evidence standards and clinical claim boundaries. We contribute ACP-Bench, a framework that adapts existing harness systems from computational pathology support toward ACP workflow capability. ACP-Bench evaluates 41 pathology workflow tasks, including 24 biomarker, 7 morphology and 10 prognosis tasks spanning 6 body-system groups and 9 endpoint families. The benchmark evaluates 9 models and 3 harness groups (Claude Code, Codex and Open Code), yielding 369 complete trajectories. ACP-Bench evaluates each trajectory across workflow execution, diagnostic performance and clinical-boundary alignment, combining expert-adjudicated process audits, diagnostic assessment and pathologist-validated safety review. Across evaluated systems, workflow initiation, task interpretation and diagnostic reporting were more mature than tool-bound execution, result binding and reflective workflow revision, and formal end-to-end completion remained rare. ACP-Bench provides a reusable standard for auditing whether agentic systems can operationalize pathology workflows before claims of reliable clinical autonomy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。