arXiv:2602.15902cs.CLcs.AI2026-02被引 27

让大模型瞬间记住长文档内容,推理更快更省内存。

Doc-to-LoRA: Learning to Instantly Internalize Contexts

  • 用轻量级超网络快速生成适配器,实现单次前向传播的上下文压缩。
  • 在超过原模型4倍长度的长文本中零样本准确率达近完美。
  • 适合需要频繁更新知识或个性化交互的场景,显著降低内存与延迟。

长输入序列是大语言模型进行上下文学习、文档理解与多步推理的核心。然而,Transformer 的二次注意力开销导致推理内存占用高且缓慢。虽然上下文蒸馏(CD)可将信息迁移到模型参数中,但每次提示都需训练的蒸馏方式因成本和延迟过高而不切实际。为此,我们提出 Doc-to-LoRA(D2L),一种轻量级超网络,能通过元学习在单次前向传播中近似完成上下文蒸馏。给定未见过的提示,D2L 为目标模型生成一个 LoRA 适配器,使后续查询无需重新读取原始上下文,从而降低推理时的延迟和键值缓存内存消耗。在长文本找针任务中,D2L 成功将上下文映射为存储关键信息的适配器,在超出目标模型原生上下文窗口4倍以上的序列长度下实现接近完美的零样本准确率。在真实世界问答数据集上,有限算力条件下,D2L 性能优于标准上下文蒸馏,同时大幅减少峰值内存占用与更新延迟。我们预期 D2L 可推动大模型的快速适应,实现频繁的知识更新与个性化聊天行为。

原文摘要 · Abstract (English)

Long input sequences are central to in-context learning, document understanding, and multi-step reasoning of Large Language Models (LLMs). However, the quadratic attention cost of Transformers makes inference memory-intensive and slow. While context distillation (CD) can transfer information into model parameters, per-prompt distillation is impractical due to training costs and latency. To address these limitations, we propose Doc-to-LoRA (D2L), a lightweight hypernetwork that meta-learns to perform approximate CD within a single forward pass. Given an unseen prompt, D2L generates a LoRA adapter for a target LLM, enabling subsequent queries to be answered without re-consuming the original context, reducing latency and KV-cache memory consumption during inference of the target LLM. On a long-context needle-in-a-haystack task, D2L successfully learns to map contexts into adapters that store the needle information, achieving near-perfect zero-shot accuracy at sequence lengths exceeding the target LLM's native context window by more than 4x. On real-world QA datasets with limited compute, D2L outperforms standard CD while significantly reducing peak memory consumption and update latency. We envision that D2L can facilitate rapid adaptation of LLMs, opening up the possibility of frequent knowledge updates and personalized chat behavior.

上下文学习LoRA长文本处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。