无需训练，用稀疏编码分离风格与语义，实现个性化文本生成
From Profiles to Steering Vectors: Global Sparse Priors and Local Semantic Calibration for Personalized Text Generation
- 用稀疏自编码器提取全局用户风格先验，结合局部对比向量
- 在不同层注入风格向量，提升生成内容的上下文感知性
- 适合需要快速适配用户风格且不改模型的场景
个性化文本生成需从历史数据中捕捉用户写作风格。现有基于检索、参数高效微调或激活引导的方法或引入推理与存储开销，或难以分离风格信号与语义内容。我们提出GLASS，一种无需训练的个性化生成框架，通过全局-局部激活引导与稀疏先验实现。GLASS利用稀疏自编码器（SAE）从历史回复中提取全局用户风格先验，并在聚类的交互场景上构建局部对比风格向量。推理时，联合将全局与局部向量注入模型不同层，实现无检索、无参数更新的上下文感知个性化生成。在LaMP和LongLaMP上的实验表明，GLASS在ROUGE指标和LLM-as-judge评估中均优于检索、微调及引导基线。进一步分析显示，基于SAE的表征对主题和长度变化更鲁棒，表明其对风格信息与语义残差具有更好解耦能力。
原文摘要 · Abstract (English)
Personalized text generation requires models to capture user-specific writing styles from historical data. Existing approaches based on retrieval, parameter-efficient fine-tuning, or activation steering either introduce inference and storage overhead or struggle to separate stylistic signals from semantic content. We propose GLASS, a training-free framework for personalized generation via Global-Local Activation Steering with Sparse priors. GLASS uses sparse autoencoders to extract a global user-style prior from historical responses and constructs local contrastive style vectors over clustered interaction scenarios. During inference, it jointly injects global and local vectors into different model layers, enabling context-aware personalization without retrieval or parameter updates. Experiments on LaMP and LongLaMP show that GLASS outperforms retrieval-, fine-tuning-, and steering-based baselines across ROUGE metrics and LLM-as-judge evaluations. Further analyses show that SAE-based representations are more robust to topic and length shifts, suggesting better disentanglement of stylistic information from semantic residue.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。