针对金融领域量化大模型,提出稳定记忆编辑框架,缓解持续更新中的知识崩溃问题。
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance

- 采用低秩适配器与闭环控制,限制修改范围并监控稳定性退化
- 在4比特量化下减少11%-17%知识退化,测试泛化率提升至28%
- 适合资源受限的金融场景持续知识更新,尤其关注稳定性
部署于动态金融环境的大语言模型面临核心挑战:市场状况、监管规则和企业信息持续变化时,如何保持事实准确性。尽管4比特量化支持高效部署,但严重限制了连续记忆编辑的可行性:现有方法在此条件下出现灾难性性能下降,陷入“量化稳定性危机”。本文提出CACHE-UK(面向英国金融领域的上下文自适应持续混合编辑器),一个专为特定领域、量化大模型设计的稳定性感知记忆编辑框架。该框架集成三个组件:将修改限制在低秩适配器子空间的秩-1 LoRA扰动机制,基于金融领域优先级的内容自适应编辑强度模块,以及追踪“退化债务”的闭环稳定性控制器,防止连续更新中的灾难性遗忘。在4比特量化下的OpenLLaMA-3B模型上,使用包含88,021篇文档的英国金融语料库进行评估,相比适配基线,CACHE-UK在相同4比特约束下将知识退化降低11%-17%,表现最稳健;同时达到本设置中最高的测试成功率(泛化率28%),比最强基线提升6个百分点。结果表明,稳定性感知编辑可改善资源受限金融大模型的事实维护能力,尽管绝对泛化率仍较低。
原文摘要 · Abstract (English)
Large Language Models (LLMs) deployed in dynamic financial environments face a critical challenge: maintaining factual accuracy as market conditions, regulations, and corporate facts change continuously. While 4-bit quantization enables efficient deployment, it severely limits the viability of sequential memory editing: existing methods undergo catastrophic performance degradation under this "quantization stability crisis." We introduce CACHE-UK (Contextual Adaptive Continual Hybrid Editor for UK Finance), a stability-aware memory editing framework specifically designed for domain-specific, quantized LLMs. CACHE-UK integrates three components: a rank-1 LoRA perturbation mechanism that confines edits to the low-rank adapter subspace, a financial domain prioritization module for content-adaptive edit strength, and a closed-loop Stability Controller that tracks "degradation debt" to prevent catastrophic forgetting across sequential updates. Evaluated on a 4-bit quantized OpenLLaMA-3B model with a curated UK financial corpus of 88,021 documents, CACHE-UK reduces knowledge degradation by 11-17% relative to adapted baselines under identical 4-bit constraints -- its most robust effect -- while attaining the highest test success (generalization) rate observed in our setting (28%, a 6 percentage point improvement over the strongest adapted baseline). These results indicate that stability-aware editing can improve factual maintenance in resource-constrained financial LLM deployments, though absolute generalization rates remain low.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。