用聚类压缩设备端大模型的记忆，省空间还更准。
Clustering-driven Memory Compression for On-device Large Language Models
- 按记忆相似性分组后合并，减少冗余。
- 相同上下文预算下，记忆量减少40%以上且生成更优。
- 适合资源受限的手机等设备部署。
大语言模型常依赖用户历史交互提炼的个性化记忆来生成内容。将记忆直接拼接至输入提示是常见做法，但会迅速耗尽设备端模型有限的上下文长度。通过平均压缩记忆虽能缓解上下文增长，却因异构记忆间的语义冲突导致性能下降。本文提出基于聚类的记忆压缩策略，在拼接前按相似性分组并合并记忆，兼顾上下文效率与个性化质量。实验表明，该方法显著降低记忆令牌数量，优于朴素平均或直接拼接等基线策略。在固定上下文预算下，聚类驱动的合并生成更紧凑的记忆表示，并持续提升生成质量。
原文摘要 · Abstract (English)
Large language models (LLMs) often rely on user-specific memories distilled from past interactions to enable personalized generation. A common practice is to concatenate these memories with the input prompt, but this approach quickly exhausts the limited context available in on-device LLMs. Compressing memories by averaging can mitigate context growth, yet it frequently harms performance due to semantic conflicts across heterogeneous memories. In this work, we introduce a clustering-based memory compression strategy that balances context efficiency and personalization quality. Our method groups memories by similarity and merges them within clusters prior to concatenation, thereby preserving coherence while reducing redundancy. Experiments demonstrate that our approach substantially lowers the number of memory tokens while outperforming baseline strategies such as naive averaging or direct concatenation. Furthermore, for a fixed context budget, clustering-driven merging yields more compact memory representations and consistently enhances generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。