为软件工程智能体设计跨任务记忆，但实验发现记忆反而带来干扰噪声。
From Knowledge to Noise: CTIM-Rover and the Pitfalls of Episodic Memory in Software Engineering Agents
- 基于经验学习构建跨任务实例记忆，支持全局与项目级知识留存
- 在多个配置下表现均不如基线模型，性能反降
- 适合关注智能体记忆机制副作用的研究者
我们提出CTIM-Rover，一个基于AutoCodeRover（Zhang et al., 2024）的软件工程智能体，引入一种通用且项目级的跨任务实例记忆（CTIM）。现有开源SE代理多依赖ReAct、Reflexion或Code-Act等推理框架，这些方法在单次任务后会丢弃长期记忆。由于修复缺陷需理解整个代码库的关联位置，我们假设SE特别适合利用CTIM。为此，我们借鉴经验学习框架ExpeL（Zhao et al., 2024），采用混合专家（MoEs）结构构建通用与项目级双层记忆。然而实验表明，无论何种配置，CTIM-Rover均未超越AutoCodeRover。分析显示，记忆中无关项或示例轨迹带来的噪声是性能下降的主要原因。
原文摘要 · Abstract (English)
We introduce CTIM-Rover, an AI agent for Software Engineering (SE) built on top of AutoCodeRover (Zhang et al., 2024) that extends agentic reasoning frameworks with an episodic memory, more specifically, a general and repository-level Cross-Task-Instance Memory (CTIM). While existing open-source SE agents mostly rely on ReAct (Yao et al., 2023b), Reflexion (Shinn et al., 2023), or Code-Act (Wang et al., 2024), all of these reasoning and planning frameworks inefficiently discard their long-term memory after a single task instance. As repository-level understanding is pivotal for identifying all locations requiring a patch for fixing a bug, we hypothesize that SE is particularly well positioned to benefit from CTIM. For this, we build on the Experiential Learning (EL) approach ExpeL (Zhao et al., 2024), proposing a Mixture-Of-Experts (MoEs) inspired approach to create both a general-purpose and repository-level CTIM. We find that CTIM-Rover does not outperform AutoCodeRover in any configuration and thus conclude that neither ExpeL nor DoT-Bank (Lingam et al., 2024) scale to real-world SE problems. Our analysis indicates noise introduced by distracting CTIM items or exemplar trajectories as the likely source of the performance degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。