发现编码智能体的潜在编程前瞻能力，能提前预测代码修改结果。
Latent Programming Horizons in Coding Agents
- 用线性探测分析语言模型隐状态，解码代码是否通过测试
- 可提前25步预测未来修改结果，准确率高于随机水平
- 跨数据集迁移有效，为可解释性研究提供新方向
一个解决软件工程任务的编码智能体需经历数十步推理、编辑代码和运行测试，但对其内部语言模型如何表征程序尚不清楚。我们发现，在编码智能体操作下，语言模型的残差流线性编码了程序的演化属性：对隐藏状态进行逻辑回归探测，可解码当前代码是否通过解析、通过测试套件、减少失败测试数及引入回归，最大AUC达0.83（覆盖两个模型与两个基准）。更令人意外的是，这些表征领先于智能体自身修改行为：训练用于预测未来修改结果（在实际写入磁盘前）的探测器，性能在约25步前仍显著优于随机水平。我们称此为智能体的潜在编程视野。作为外部有效性验证，探测器无需重训即可跨基准迁移。积极结果呼吁进一步开展编码智能体机制可解释性研究。
原文摘要 · Abstract (English)
A coding agent solving a software-engineering task spends dozens of steps reasoning, editing code, and running tests, yet little is known about what the underlying language model internally represents about the program it is working on. We show that the residual streams of language models under coding agents linearly encode properties of the evolving program: a logistic-regression probe on hidden states is able to decode whether the current code parses, passes its test suite, reduces the number of failing tests, and introduces regressions, reaching AUC up to 0.83 for correctness across two models and two benchmarks. Our second finding is more surprising: these representations run ahead of the agent's own edits. Probes trained to predict the outcome of future edits (before they are materialized and written on disk) achieve performance above chance up to roughly 25 steps in advance. We call this the agent's latent programming horizon. As a proof of external validity, we show that the probes transfer across benchmarks without retraining. Our positive results open calls for more research in mechanistic interpretability of coding agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。