用动态软提示表征用户记忆，高效个性化大模型。
Latent Personal Memory: Represent personal memory as dynamic soft prompts
- 将用户记忆建模为可解释的潜在槽矩阵，通过交叉注意力生成动态提示。
- 在PersonaMem上比LoRA高8.8%准确率，比提示调优高54.4%，显存降低64倍。
- 参数量仅需LoRA的1/120，适合长上下文场景，尤其适用于资源受限设备。
个性化大型语言模型（LLM）需要以计算高效、可扩展且兼容冻结基础模型的方式编码长期、用户特定的行为模式。我们提出潜空间个人记忆（LPM），一种可扩展框架，将用户历史表示为一个紧凑、持久的N个潜在槽矩阵，具备可解释性。一个共享的交叉注意力投影网络将这些槽映射为输入相关的动态软提示，并前置到冻结LLM的输入中。我们在PersonaMem v1和LoCOMO基准上评估了基于Qwen3-1.7B、4B和8B骨干模型的LPM。结果表明，在PersonaMem v1上，LPM相比LoRA和提示调优分别提升最高8.8%和54.4%的整体准确率，同时减少超过64倍的KV缓存使用。在LoCoMo上，LPM达到与LoRA相当的准确率，但训练参数减少120倍。我们还发现，随着上下文长度增加，LPM的效率优势更显著，在128K上下文长度下优于全上下文方法。
原文摘要 · Abstract (English)
Personalizing large language models (LLMs) requires encoding long-term, user-specific behavioral patterns in a way that is computationally efficient, scalable, and compatible with a frozen base model. We present Latent Personal Memory (LPM), a scalable framework that represents user-specific history as a compact, persistent matrix of N latent slots, that are interpretable. A shared cross-attention projection network maps these slots into dynamic, input-conditioned soft prompts that are prepended to the input of a frozen LLM. We evaluate LPM on PersonaMem v1 and LoCOMO benchmarks across Qwen3-1.7B, 4B, and 8B backbones. Results demonstrate that LPM outperforms LoRA and Prompt Tuning by up to 8.8% and 54.4% in overall accuracy respectively on PersonaMem v1, while reducing KV-cache usage by over 64x. On LoCoMo, LPM matches LoRA accuracy with 120x fewer trainable parameters. We also show that the efficiency of LPM grows with context length and outperforms full-context at 128K context length.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。