大模型生成时可并行处理多任务,分散注意力未必是错误。
Can Large Language Models "Hyper-Thread"?
- 通过并发任务加载,让每步生成同时处理多个子任务。
- 在AIME 2025上准确率最高,输出长度相近但注意力更分散。
- 适合关注推理效率与生成机制优化的研究者。
大型语言模型通常逐个生成标记,但能否在生成每个标记时并行执行多个任务?更广泛的注意力分配可能为此提供机制。现有推理扩展方法主要依赖更长生成、更多采样或额外验证阶段,而注意力分散常被视为干扰或错误信号。因此,串行生成中的任务并行性仍被忽视。本文提出模型超线程假说,并通过共享状态的多任务协作实验进行验证。设计三种条件(基线、串行功能调度、并发功能加载),基于准确率、输出标记分布和注意力指标评估其收益与代价。在AIME 2025开发集上,并发功能加载达到最高准确率。相较于串行调度,其典型输出长度相似,多数问题更短,且注意力更分散、任务相关覆盖更高,尽管输出长度尾部更重。单步内并行性及其因果机制仍需直接验证。结果表明,更分散注意力可与更高准确率共存,为超线程假说提供了初步行为与相关证据。这提示推理扩展应从‘生成更多标记’转向‘每步承载更多任务’,开辟提升推理性能的新路径。
原文摘要 · Abstract (English)
Large language models generate tokens sequentially, but can they execute multiple tasks concurrently while forming each token? Broader attention allocation may provide a mechanism for such task concurrency. Existing approaches to scaling inference primarily rely on longer generations, more samples, or additional verification stages, while attention dispersion is often treated as a signal of interference or error. Task concurrency within serial generation therefore remains underexplored. We propose the Model Hyper-Threading Hypothesis and evaluate its predictions using multiple coordinated tasks that share state within the same problem. We design three conditions (Baseline, Serial Functional Scheduling, and Concurrent Functional Loading) and evaluate their benefits and costs using accuracy, output-token distributions, and attention metrics. On an AIME 2025 development set, Concurrent Functional Loading achieves the highest accuracy. Relative to Serial Functional Scheduling, its typical output length is similar and it is shorter on most problems, while exhibiting greater attention dispersion and higher task-relevant coverage, albeit with a heavier output-length tail. Within-step concurrency and its causal mechanism still require direct tests. Our results show that more dispersed attention can coexist with higher accuracy, providing preliminary behavioral and correlational evidence for the hyper-threading hypothesis. These findings motivate a shift in perspective on inference scaling from "generating more tokens" toward "having each generation step carry more tasks," pointing to a new avenue for improving reasoning performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。