让AI Agent通过反馈自我优化指令,无需重训也能持续进化。
Instruction-Level Weight Shaping: A Framework for Self-Improving AI Agents
- 用用户反馈和反思机制动态调整系统指令,实现无中断自更新
- 企业支持场景下吞吐量提升2.4至5倍,幻觉减少约80%
- 适合需要快速适应新知识的法律、医疗等动态领域
大型语言模型在预训练后基本保持静态,新增或变化的知识通常依赖检索增强生成(RAG)或微调。RAG增加延迟与工程成本,且难以有效整合事实;提示工程脆弱易冲突;微调成本高且有灾难性遗忘风险。我们提出指令级权重塑形(ILWS):由人工精心设计的系统指令作为外部可审计的伪参数,每次会话后通过反思和用户反馈进行更新。反思引擎分析对话轨迹,诊断推理成败,并提出针对指令、用户偏好和工具的类型化增量ΔK=(ΔS,ΔU,ΔT)。增量采用版本控制,通过滑动窗口的1-5星评分评估,首次失败自动修复,重复失败则回滚。当编辑预算超过阈值时,代理将合成带评分加权的数据集,将成熟化的指令空间改进蒸馏为参数,实现无需停机的从提示空间到权重空间的转化。ILWS显式揭示了上下文在Transformer块中引起的低秩塑形,保留治理能力,消除每轮调用的检索需求。在企业支持场景中,吞吐量提升2.4-5.0倍,审计级幻觉减少约80%;在Adobe Commerce Cloud的L0支持概念验证中,每小时处理工单量提升4-5倍,平均处理时间降低约80%,支持自主指令更新与可选工具合成。由于ILWS在蒸馏前始终运行于指令层,可推广至需自适应推理、工具创建和低延迟部署的动态领域(如法律、医学、工程)。
原文摘要 · Abstract (English)
Large language models (LLMs) are fluent but largely static after pre-training; new or shifting knowledge is typically added with retrieval-augmented generation (RAG) or fine-tuning. RAG raises latency and engineering overhead and often fails to integrate facts; prompt engineering is brittle and can conflict with prior knowledge; fine-tuning is costly and risks catastrophic forgetting. We propose Instruction-Level Weight Shaping (ILWS): curated system instructions act as external, auditable pseudo-parameters updated after each session via reflection and user feedback. A Reflection Engine inspects conversation traces, diagnoses reasoning successes and failures, and proposes typed deltas $ΔK=(ΔS,ΔU,ΔT)$ over instructions, user preferences, and tools. Deltas are version-controlled, evaluated with a sliding window of 1-5 star ratings, auto-repaired on first failure, and rolled back on repeated failure. When an edit budget crosses a threshold, the agent compiles a rating-weighted synthetic set and distills matured instruction-space gains into parameters, converting prompt-space improvements into weight-space without downtime. ILWS makes explicit the low-rank shaping induced by context in transformer blocks, preserves governance, and removes per-call retrieval. In enterprise support it increased throughput 2.4-5.0x and cut audited hallucinations by about 80% versus a frozen baseline. In an Adobe Commerce Cloud proof of concept "L0 Support", it achieved 4-5x more tickets per hour and about 80% lower time per ticket, with autonomous instruction updates and optional tool synthesis. Because ILWS operates at the instruction layer until controlled distillation, it generalizes to dynamic domains (legal, medical, engineering) requiring adaptive reasoning, tool creation, and low-latency deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。