研究发现干预时机判断不可靠，现有方法易失效。
The Saturation Trap and the Subjectivity of Intervention Timing: Why Affect-Based Triggers and LLM Judges Fail to Time Interventions on Autonomous Agents
- 用情感动态模型检测智能体干预时机，发现阈值触发器会因状态饱和持续误报。
- 小模型判别器完全不触发，大模型需全轨迹上下文且准确率仅0.17-0.40。
- 人类标注者对干预位置和类型分歧严重，可靠性接近随机。
随着自主AI智能体从对话系统转向长时序软件执行，运行时安全层需决定何时中断其行为。本文使用18维情感动态引擎HEART作为诊断工具，评估四类干预触发机制——绝对状态阈值、复合状态-动作模式、正则表达式特征提取、零样本大模型判官——在SWE-bench-Verified调试轨迹上的人工标注干预点表现。结果发现：第一，存在状态饱和陷阱：智能体持续困难时无恢复信号，模型化挫败感迅速达上限并保持，使状态阈值触发器从瞬时检测变为近恒定激活，5条轨迹中39%-83%动作触发；第二，大模型判官存在能力与上下文下限：gpt-5.4-mini始终不触发，前沿及跨厂商模型仅在完整轨迹上下文中勉强脱离零触发，且最高F1仅为0.17-0.40，代价高达90倍；第三，最核心问题在于人工标注目标不可复现：三位训练标注者使用同一标准对56步轨迹的干预位置一致性仅略高于随机（Krippendorff's alpha = +0.047；最佳成对Cohen's kappa = +0.349），干预类型更无共识（暂停退化；澄清低于随机；反思仅alpha = +0.226）。结论表明干预时机是低可靠性构念，单标注者F1不适合作为优化目标。贡献在于联合映射了人类标注可靠性、四类检测架构、跨模型大模型判官性能与重现的饱和效应，而非单一检测器精度。
原文摘要 · Abstract (English)
As autonomous AI agents move from conversational systems to long-horizon software execution, runtime safety layers that decide when to interrupt an agent have become essential. We study this timing problem using a continuous 18-dimensional affective-dynamics engine (HEART) as a diagnostic probe, evaluating four intervention trigger families - absolute state thresholds, composite state-action patterns, regex reasoning-feature extraction, and zero-shot LLM-as-judge - against human-annotated intervention points on SWE-bench-Verified debugging traces. We report three findings. First, a State Saturation Trap: agents show no recovery signal under sustained difficulty, so modeled frustration quickly crosses the threshold and stays at its maximum, converting threshold-on-state triggers from moment detectors into near-constant indicators that fire on 39-83% of actions across five trajectories. Second, a capability-and-context floor for LLM judges: a small model (gpt-5.4-mini) never fires, while frontier and cross-vendor models escape the zero-firing floor only with full-trajectory context, and even then reach only F1 0.17-0.40 at up to 90x the cost. Third, and most importantly, the supervised target is not reproducible among humans: three trained annotators using one rubric on a 56-action trajectory agree on where to intervene only slightly above chance (location Krippendorff's alpha = +0.047; best pairwise Cohen's kappa = +0.349) and not at all on intervention type (pause degenerate; clarify below chance; reflect only alpha = +0.226). We conclude that intervention timing is a low-reliability construct, making single-annotator F1 an unsuitable optimization target. Our contribution is the joint mapping of this problem across human inter-rater reliability, four detector architectures, a cross-model LLM-judge sweep, and a reproduced saturation effect, rather than any single detector's accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。