用激活空间向量分离用户文风，生成更精准且省存储的个性化文本。
Personalized Text Generation with Contrastive Activation Steering
- 将用户文风解耦为激活空间中的向量，避免内容与风格混淆
- 生成效果提升8%，存储需求比参数微调少1700倍
- 无需训练、不依赖检索，适合高并发个性化生成场景
个性化文本生成旨在从用户历史文本中推断其写作风格偏好，并生成忠实反映这些特征的内容。现有方法主要采用检索增强生成（RAG）和参数高效微调（PEFT）两种范式。然而，这些方法存在两大局限：（1）历史文本中内容语义与风格模式纠缠，难以准确建模用户写作风格；（2）可扩展性差，RAG因检索操作导致推理延迟高，而PEFT需为每位用户存储大量参数。为此，我们提出StyleVector——一种无需训练的框架，将个性化写作风格解耦并表示为大语言模型激活空间中的向量，实现推理时无须昂贵检索或参数存储的风格引导生成。全面实验表明，该框架在个性化生成上取得8%的相对性能提升，同时相比PEFT方法将存储需求降低1700倍。
原文摘要 · Abstract (English)
Personalized text generation aims to infer users' writing style preferences from their historical texts and generate outputs that faithfully reflect these stylistic characteristics. Existing solutions primarily adopt two paradigms: retrieval-augmented generation (RAG) and parameter-efficient fine-tuning (PEFT). While these approaches have advanced the field, they suffer from two critical limitations: (1) the entanglement of content semantics and stylistic patterns in historical texts impedes accurate modeling of user-specific writing preferences; and (2) scalability challenges arising from both RAG's inference latency by retrieval operations and PEFT's parameter storage requirements for per user model. To overcome these limitations, we propose StyleVector, a training-free framework that disentangles and represents personalized writing style as a vector in LLM's activation space, enabling style-steered generation during inference without requiring costly retrieval or parameter storage. Comprehensive experiments demonstrate that our framework achieves a significant 8% relative improvement in personalized generation while reducing storage requirements by 1700 times over PEFT method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。