让大模型边听边想边答，无需训练即可实现实时交互。
Asynchronous Reasoning: Training-Free Interactive Thinking LLMs
- 利用位置嵌入特性实现思考、听输入、输出并行进行。
- 首非思考词延迟从分钟级降至5秒内，总延迟降低12倍。
- 适合语音助手、机器人等需实时响应的场景。
当前许多先进大模型通过思考提升能力，但需完成思考后才能回应，影响交互性。真实场景如语音或具身助手要求模型能实时响应并适应新信息，与顺序式交互不兼容。人类可一边听一边思考一边作答。本文提出一种无需额外训练的方法，利用位置嵌入特性，使具备推理能力的大模型在生成过程中同步进行思考、接收输入和输出。我们在数学、常识与安全推理任务上评估该方法，结果表明模型能在保持高准确率的同时，将首个非思考词的延迟从分钟级降至≤5秒，整体延迟最高降低12倍。
原文摘要 · Abstract (English)
Many state-of-the-art LLMs are trained to think before giving their answer. Reasoning can greatly improve language model capabilities, but it also makes them less interactive: given a new input, a model must stop thinking before it can respond. Real-world use cases such as voice-based or embodied assistants require an LLM agent to respond and adapt to additional information in real time, which is incompatible with sequential interactions. In contrast, humans can listen, think, and act asynchronously: we begin thinking about the problem while reading it and continue thinking while formulating the answer. In this work, we augment LLMs capable of reasoning to operate in a similar way without additional training. Our method uses the properties of positional embeddings to enable LLMs built for sequential generation to simultaneously think, listen, and write outputs. We evaluate our approach on math, commonsense, and safety reasoning: it allows models to generate accurate thinking-augmented answers while reducing time to first non-thinking token from minutes to ${\le}$ 5s and the overall delays by up to $12{\times}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。