为大模型代理设计精准异常定位系统，提升运行可靠性。
TrajAD: Trajectory Anomaly Detection for Trustworthy LLM Agents
- 构建轨迹异常检测任务，聚焦中间执行过程审计。
- 通用大模型零样本下定位准确率低，泛化能力不等于过程可靠。
- 提出专用验证器TrajAD，细粒度监督显著提升检测效果。
我们解决运行时轨迹异常检测问题，这是实现可信大模型代理的关键能力。现有安全措施主要关注静态输入输出过滤，但我们认为确保大模型代理可靠性需审计中间执行过程。本文提出轨迹异常检测任务，目标不仅是检测，更是精准定位异常，以支持高效回滚重试。为此，我们构建了基于扰动与补全策略合成的TrajBench数据集，涵盖多样化的流程异常。在该基准上评估模型过程监督能力，发现通用大模型即使采用零样本提示也难以识别和定位异常，表明泛化能力无法自动转化为过程可靠性。为此，我们提出TrajAD，一种通过细粒度过程监督训练的专用验证器。实验表明，该方法优于基线，证明专用监督对构建可信代理至关重要。
原文摘要 · Abstract (English)
We address the problem of runtime trajectory anomaly detection, a critical capability for enabling trustworthy LLM agents. Current safety measures predominantly focus on static input/output filtering. However, we argue that ensuring LLM agents reliability requires auditing the intermediate execution process. In this work, we formulate the task of Trajectory Anomaly Detection. The goal is not merely detection, but precise error localization. This capability is essential for enabling efficient rollback-and-retry. To achieve this, we construct TrajBench, a dataset synthesized via a perturb-and-complete strategy to cover diverse procedural anomalies. Using this benchmark, we investigate the capability of models in process supervision. We observe that general-purpose LLMs, even with zero-shot prompting, struggle to identify and localize these anomalies. This reveals that generalized capabilities do not automatically translate to process reliability. To address this, we propose TrajAD, a specialized verifier trained with fine-grained process supervision. Our approach outperforms baselines, demonstrating that specialized supervision is essential for building trustworthy agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。