用连续嵌入压缩上下文，发现大模型在复杂编码任务中仍会失败。
On Problems of Implicit Context Compression for Software Engineering Agents
- 将上下文转为连续嵌入，提升信息存储密度。
- 单次任务表现良好，但多步骤编码任务成功率下降。
- 揭示了隐式上下文压缩在复杂任务中的局限性。
基于大语言模型的软件工程代理面临关键瓶颈：上下文长度限制导致在复杂、长周期任务中失败。一种有前景的解决方案是将上下文编码为连续嵌入而非离散标记，从而实现更密集的信息存储。我们应用了最近提出的In-Context Autoencoder方法。尽管该方法在单次任务的通用知识和代码理解上表现良好，但我们的实验表明，其在多步骤代理编码任务中表现不佳。本文探讨了这一现象，并分析可能导致失败的潜在因素。
原文摘要 · Abstract (English)
LLM-based Software Engineering agents face a critical bottleneck: context length limitations cause failures on complex, long-horizon tasks. One promising solution is to encode context as continuous embeddings rather than discrete tokens, enabling denser information storage. We apply the recently proposed In-Context Autoencoder for this purpose. While the method performs well on single-shot common-knowledge and code-understanding tasks, our experiments demonstrate that it fails on multi-step agentic coding tasks. In this paper, we explore this phenomenon and discuss possible factors contributing to this failure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。