为强化学习编程代理设计可审计的记忆系统，提升代码生成安全性。
Feedback-Normalized Developer Memory for Reinforcement-Learning Coding Agents: A Safety-Gated MCP Architecture
- 基于MCP协议构建本地优先记忆架构，动态管理上下文决策
- 在200个测试用例中实现80%决策准确率与100%难例抑制
- 适合关注代码安全与可追溯性的自动化开发系统开发者
大型语言模型编程代理在软件工程长周期任务中需处理仓库、终端、测试和执行轨迹。持久化记忆虽有用，但静态向量存储或通用检索增强生成（RAG）不足以应对强化学习（RL）代码开发，因微小细节可能改变贝尔曼目标、终止掩码、梯度流或验证结论。本文提出RL开发者记忆，一种本地优先、原生支持模型上下文协议（MCP）的开发者记忆架构。该系统将记忆选择视为可记录的上下文决策过程：issue_match按候选项排名并记录遥测数据，issue_feedback将原始标签映射为有界奖励，issue_record_resolution将已验证解决方案与早期检索事件关联。确定性排序器持续部署，而上下文老虎机残差策略以影子模式运行，仅通过保守的离策略评估（OPE）门控影响蜜罐行为。RL/控制记忆需理论到代码的元数据及评审授权治理。系统在包含RL算法缺陷、硬负例、评审授权案例和低风险失败的200例确定性基准上评估。相同提交对比下，确定性控制与完整影子/OPE均达80.0%预期决策准确率和100.0%硬负例抑制；完整配置仅增加学习遥测，无准确率提升。静态验证通过11/11项检查；动态集成通过10/10案例。证据表明：主动学习策略部署、官方客户端MCP互操作性尚未支持，全配置实时延迟上升，仍有40个残余非RL失败。贡献在于可审计的记忆控制架构与明确的主张边界，而非通用编码代理改进。
原文摘要 · Abstract (English)
Large language model (LLM) coding agents increasingly operate over repositories, terminals, tests, and execution traces across long software-engineering episodes. Persistent memory is useful, but static vector stores or generic retrieval-augmented generation (RAG) are insufficient for reinforcement-learning (RL) code development, where small details can alter Bellman targets, terminal masks, gradient flow, or validation claims. This paper presents RL Developer Memory, a local-first, Model Context Protocol (MCP)-native developer-memory architecture for RL coding agents. It treats memory selection as a logged contextual decision process: issue_match ranks candidates and records telemetry, issue_feedback maps raw labels to bounded rewards, and issue_record_resolution links verified resolutions to earlier retrieval events. A deterministic ranker remains deployed, while a contextual-bandit residual policy runs in shadow mode and can affect canary behavior only through conservative off-policy-evaluation (OPE) gates. RL/control memories require theory-to-code metadata and review-gated governance. The system is evaluated on a deterministic 200-case benchmark with RL algorithm bugs, hard negatives, review-gated RL/control cases, and low-risk failures. In the same-commit comparison, deterministic control and full shadow/OPE both achieve 80.0% expected-decision accuracy and 100.0% hard-negative suppression; the full configuration adds learning telemetry rather than accuracy gain. Static validation passed 11/11 checks; dynamic integration passed 10/10 cases. The evidence reports limits: active learned-policy deployment and official-client MCP interoperability are unsupported, live full-configuration latency regresses, and 40 residual non-RL failures remain. The contribution is an auditable memory-control architecture with explicit claim boundaries, not a universal coding-agent improvement claim.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。