用基因序列方法分析大模型智能体行为,发现高风险模式并提升成功率6.2%。
Your Agent Has a Genome: Sequence-Level Behavioral Analysis and Runtime Governance of LLM-Powered Autonomous Agents

- 将智能体行为编码为XEPV四字母序列,类比基因分析
- 发现P-X-P模式使成功率降低10.4%,验证环节仅2.1%触发
- 设计运行时干预系统Governor,任务成功提升6.2%,节省44%token
我们提出基础序列分析框架,将大语言模型驱动的自主智能体运行行为编码为由X(探索)、E(执行)、P(规划)、V(验证)组成的紧凑符号序列。基于生产环境中8天内收集的347条真实执行轨迹,采用n-gram挖掘、马尔可夫转移矩阵和点双列相关分析,发现:(1) 三元组P-X-P是唯一统计显著的高风险模式,使成功率下降10.4%;(2) P比率是成功的最强负向预测因子(r=-0.256, p<0.0001);(3) 执行到验证的转移概率仅为2.1%,表明系统存在普遍验证缺陷。据此设计了三层运行时干预系统Governor,包含规则引擎、统计累积器与卡方阈值自适应模块。自然前后对比评估显示(N=101 vs. N=246),Governor使任务成功率绝对提升6.2%,同时平均令牌消耗减少44%。为验证跨系统通用性,将XEPV编码应用于SWE-bench上2000条公开的SWE-agent轨迹,确认探索循环与执行-验证断层在独立系统中重现。论文还提出六项研究方向,并开源工具包以支持复现。
原文摘要 · Abstract (English)
We propose Base Sequence Analysis, a framework that encodes the runtime behavior of LLM-powered autonomous agents into compact symbolic sequences using a four-letter alphabet: X (Explore), E (Execute), P (Plan), and V (Verify). Drawing an analogy to genomic sequence analysis, we apply n-gram pattern mining, Markov transition matrices, and point-biserial correlation to 347 real-world execution traces collected from a production ReAct agent system over 8 days. Our analysis reveals that (1) the trigram P-X-P is the only statistically significant high-risk pattern, lowering success rate by 10.4%; (2) P-ratio is the strongest negative predictor of success (r=-0.256, p<0.0001); and (3) the E->V transition probability is only 2.1%, indicating a systemic verification deficit. Based on these findings, we design Governor, a three-layer runtime intervention system comprising a rule engine, a statistical accumulator, and a chi-square-based threshold adaptor. In a natural before/after deployment evaluation (N=101 vs. N=246), Governor achieves a +6.2% absolute increase in task success rate while simultaneously reducing average token consumption by 44%. To validate cross-system generality, we apply the XEPV encoding to 2,000 public SWE-agent trajectories on SWE-bench, confirming that exploration spirals and the E->V verification deficit replicate in an independent system. We outline six research directions including base sequence language models, cross-agent behavioral fingerprinting, and reward shaping, and release an open-source toolkit for reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。