用块式生成提升大模型效率，无需训练即可适配新数据。
Chunk-Distilled Language Modeling
- 一次解码生成多个词元块，突破逐词生成瓶颈。
- 在多项任务中提升生成效率与性能，且无需重新训练。
- 可灵活构建模型或领域专用知识库，适合需要快速更新的场景。
我们提出块蒸馏语言建模（CD-LM），解决当前大语言模型在文本生成中的两大问题：逐标记生成效率低，以及难以适应新数据与知识。该方法结合深度网络语言模型与简单检索模块，可在单次解码步骤中生成多标记文本块。其检索框架支持灵活构建模型或领域特定的数据存储，既可利用现有模型的内部知识，也可融入人工标注语料中的专家见解。这种灵活性使模型分布可控，无需额外训练。本文展示了CD-LM的完整形式及其在多样化下游任务中的性能指标，证明其在提升语言模型性能与效率方面的有效性。代码与数据将公开共享。
原文摘要 · Abstract (English)
We introduce Chunk-Distilled Language Modeling (CD-LM), an approach to text generation that addresses two challenges in current large language models (LLMs): the inefficiency of token-level generation, and the difficulty of adapting to new data and knowledge. Our method combines deep network-based LLMs with a straightforward retrieval module, which allows the generation of multi-token text chunks at a single decoding step. Our retrieval framework enables flexible construction of model- or domain-specific datastores, either leveraging the internal knowledge of existing models, or incorporating expert insights from human-annotated corpora. This adaptability allows for enhanced control over the language model's distribution without necessitating additional training. We present the CD-LM formulation along with performance metrics demonstrating its ability to improve language model performance and efficiency across a diverse set of downstream tasks. Code and data will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。