不更新权重,通过动态整合上下文增强大模型推理能力。
Summarize Before You Speak with ARACH: A Training-Free Inference-Time Plug-In for Enhancing LLMs via Global Attention Reallocation
- 引入自适应上下文枢纽,实时重分配注意力以优化内部计算
- 多任务实验显示性能稳定提升,推理开销小且无需训练
- 适合希望无损改进大模型推理的开发者或研究者
大型语言模型(LLMs)表现优异,但进一步提升通常依赖昂贵的训练。这推动了对后训练技术的兴趣,尤其是无需训练的推理时方法,可在不更新参数的情况下改善输出。现有方法多将模型视为黑箱,通过提示设计或重复采样、重排序/验证、搜索等输入/输出层面干预实现改进。然而,它们极少提供可即插即用的机制来干预模型内部计算。我们提出ARACH(基于自适应上下文枢纽的注意力重分配),一种无需训练的推理时插件,通过引入自适应上下文枢纽聚合信息并重新分配注意力,增强大模型能力。在多个语言建模任务上的大量实验表明,该方法能持续提升性能,仅带来微小推理开销,且无需参数更新。注意力分析进一步表明,ARACH缓解了注意力坍缩现象。结果表明,工程化模型内部计算是一种区别于提示驱动与训练驱动方法的根本性推理时策略。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve remarkable performance, yet further gains often require costly training. This has motivated growing interest in post-training techniques-especially training-free approaches that improve models at inference time without updating weights. Most training-free methods treat the model as a black box and improve outputs via input/output-level interventions, such as prompt design and test-time scaling through repeated sampling, reranking/verification, or search. In contrast, they rarely offer a plug-and-play mechanism to intervene in a model's internal computation. We propose ARACH(Attention Reallocation via an Adaptive Context Hub), a training-free inference-time plug-in that augments LLMs with an adaptive context hub to aggregate context and reallocate attention. Extensive experiments across multiple language modeling tasks show consistent improvements with modest inference overhead and no parameter updates. Attention analyses further suggest that ARACH mitigates the attention sink phenomenon. These results indicate that engineering a model's internal computation offers a distinct inference-time strategy, fundamentally different from both prompt-based test-time methods and training-based post-training approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。