arXiv:2606.08234cs.AI2026-06被引 1

让科研大模型在每一步推理中都考虑安全,防止多步操作累积成风险。

SciTrace: Trajectory-Aware Safety Reasoning for Scientific Discovery Agents

论文配图:SciTrace: Trajectory-Aware Safety Reasoning for Scientific Discovery Agents
图 1 · 摘自论文原文
  • 在每个研究步骤中嵌入安全推理,持续跟踪风险状态。
  • 能发现78.8%单步检测漏掉的复合工具链风险。
  • 适合需要高安全性的自动化科研系统开发者。

基于大语言模型的科研代理展现出自主研究的强大能力,但其安全机制与核心推理过程脱节:仅检查结果输出,而非影响生成过程的思考。这种分离导致两种失效模式:安全信号在阶段间丢失,以及看似无害的单步工具调用组合后产生有害后果,无法被单步过滤器识别。为此,我们提出 extbf{SciTrace},一个将安全推理融入科研代理全流程的框架。SciTrace结合两种互补机制:一是 extit{安全内生推理循环}(SIR),通过联合任务与安全思考,在思考、实验、写作和评审阶段持续维护累计风险状态;二是 extit{组合式工具链验证器}(CTV),在执行前进行轨迹感知的安全检查,捕捉多步序列中才暴露的风险。在涵盖六个科学领域的240个高风险研究任务和120个工具相关风险任务上评估,SciTrace在四种主干模型下均达到当前最优(SOTA)安全表现,显著提升工具调用安全性与对抗鲁棒性,同时保持科研输出质量,并发现了 extbf{78.8\%}的单步监控遗漏的组合式工具链逃逸。

原文摘要 · Abstract (English)

LLM-based scientific agents have shown strong capacity for autonomous research, yet their safety layers remain structurally divorced from core reasoning: they inspect pipeline outputs rather than shaping the deliberation that produces them. This separation opens two failure modes: safety signals accumulated at one stage are discarded before the next, and sequences of individually benign tool calls can compose into harmful outcomes that no single-step filter detects. To address these challenges, we introduce \textbf{SciTrace}, a framework that weaves safety reasoning into every stage of the scientific agent pipeline. SciTrace couples two complementary mechanisms: a \textit{Safety-Intrinsic Reasoning Loop} (SIR) that maintains a cumulative risk state across the Thinker, Experimenter, Writer, and Reviewer stages through joint task-and-safety deliberation, and a \textit{Compositional Tool-Chain Verifier} (CTV) that performs trajectory-aware safety checks before execution, catching risks that surface only across multi-step tool sequences. Evaluated on 240 high-risk research tasks and 120 tool-related risk tasks spanning six scientific domains, SciTrace achieves state-of-the-art (\textbf{SOTA}) safety among compared frameworks across four backbone models: it consistently improves tool call safety and adversarial robustness while preserving scientific output quality, and it uncovers \textbf{78.8\%} of the compositional tool-chain escapes that single-step monitors miss. The project website is available at https://opensciagent.github.io/SciTrace/.

科研代理安全推理工具链验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。