让大模型实时适应领域变化，无需重新训练。
Online Domain-aware LLM Decoding for Continual Domain Evolution
- 通过前缀树先验与基础模型在概率层融合，动态调整输出
- 在多种概念漂移场景下，ROUGE-L提升0.065，余弦相似度提高13.6%
- 适合需要持续更新知识的实时对话、客服等动态应用
大模型通常在静态领域数据上离线微调，但现实中领域知识随新法规、产品和服务不断演化。每次更新都重新训练成本过高，且数据分布随时间变化（即概念漂移）会显著降低模型性能。为此，我们提出在线领域感知解码框架ODD，通过自适应置信度调节，利用不一致性和连续性信号，在概率层融合基础大模型与前缀树先验。在多种漂移场景下的实验表明，ODD在所有句法和语义自然语言生成指标上均优于LLM-Greedy和LLM-Temp Scaled。其绝对ROUGE-L提升0.065，余弦相似度相对提升13.6%，证明了对词汇与上下文模式演变的强鲁棒性，适用于动态大模型应用场景。
原文摘要 · Abstract (English)
LLMs are typically fine-tuned offline on domain-specific data, assuming a static domain. In practice, domain knowledge evolves continuously through new regulations, products, services, and interaction patterns. Retraining or fine-tuning LLMs for every new instance is computationally infeasible. Additionally, real-world environments also exhibit temporal dynamics with shifting data distributions. Disregarding this phenomenon, commonly referred to as concept drift, can significantly diminish a model's predictive accuracy. This mismatch between evolving domains and static adaptation pipelines highlights the need for efficient, real-time adaptation without costly retraining. In response, we introduce Online Domain-aware Decoding framework (ODD). ODD performs probability-level fusion between a base LLM and a prefix-tree prior, guided by adaptive confidence modulation using disagreement and continuity signals. Empirical evaluation under diverse drift scenarios demonstrates that ODD consistently surpasses LLM-Greedy and LLM-Temp Scaled across all syntactic and semantic NLG metrics. It yields an absolute ROUGE-L gain of 0.065 and a 13.6% relative improvement in Cosine Similarity over the best baseline. These results demonstrate ODD 's robustness to evolving lexical and contextual patterns, making it suitable for dynamic LLM applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。