用词频统计在解码时控制大模型风格,轻量但极难调
Limits of n-gram Style Control for LLMs via Logit-Space Injection
- 在输出概率空间注入词频风格先验,通过调节参数控制风格
- 仅在极窄参数范围内(如lambda=0.1)提升风格和流畅度24.7%~51.4%
- 对多作者语料易崩溃,效果不如提示工程和微调,适合快速试错
大语言模型通常通过提示工程或高效微调(如LoRA)进行个性化。但写作风格难以浓缩为单一提示,而LoRA需大量训练资源。本文探索一种轻量替代方案:在解码时通过逻辑空间注入n-gram风格先验,操控冻结模型的输出。在《堂吉诃德》、CNN/DailyMail新闻标题、arXiv摘要等不同风格语料上训练1-3元语法模型,构建当前上下文匹配的加权风格概率先验。生成时,将各阶n-gram的风格对数概率按λ∈[0,1]加权叠加至模型原始logits。实验发现,在TinyLlama-1.1B上,仅当使用《堂吉诃德》语料且λ=0.1时,风格困惑度下降24.7%,基础困惑度下降51.4%;其他情况,即使小λ值也导致风格与流畅度下降,大λ值引发极端困惑度与文本崩坏。该方法虽轻量可调,但鲁棒性差,仅在极窄区间有效,总体仍弱于提示与微调。
原文摘要 · Abstract (English)
Large language models (LLMs) are typically personalized via prompt engineering or parameter-efficient fine-tuning such as LoRA. However, writing style can be difficult to distill into a single prompt, and LoRA fine-tuning requires computationally intensive training and infrastructure. We investigate a possible lightweight alternative: steering a frozen LLM with n-gram style priors injected in logit space at decoding time. We train an n-gram model on stylistically distinct corpora -- including Don Quixote, CNN/DailyMail news headlines, and arXiv abstracts -- constructing an interpolated 1-to-3-gram prior over next-token probabilities. During generation we modify the LLM's logits by adding a weighted sum of style log-probabilities from each n-gram order that matches the current context, scaled by a control parameter lambda in [0, 1]. We sweep lambda and style corpora and report style perplexity under the n-gram model, base-model perplexity as a proxy for fluency, Jensen-Shannon (JS) divergence between the original and steered token distributions, and token-overlap statistics. On TinyLlama-1.1B we identify a single narrow regime (for the Don Quixote corpus at lambda=0.1) where style perplexity improves by 24.7% and base-model perplexity improves by 51.4% relative to the frozen model. Outside this regime, and for multi-author corpora such as CNN/DailyMail and arXiv abstracts, even small nonzero lambda values generally result in worse style and fluency, and larger lambda values lead to collapse with extreme perplexities and incoherent text. Logit-space injection of n-gram style priors provides lightweight, tunable style control, but it is fragile: it operates effectively only within a narrow range of low lambda values and is consistently outperformed by prompting and LoRA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。