用双系统推理动态优化大模型推理，省算力又提速
Seer Self-Consistency: Advance Budget Estimation for Adaptive Test-Time Scaling
- 用快速推理预估答案不确定性，决定是否启用深度推理
- 比现有方法少用47%计算量，延迟降低43%且性能不降
- 适合对响应速度和成本敏感的实时大模型应用
测试时缩放能提升大语言模型的推理表现，但带来高昂的计算开销。尽管近期研究通过动态自一致性减少了令牌消耗，但仍受限于串行请求带来的高延迟。本文提出SeerSC，一种融合系统1与系统2推理的动态自一致性框架，同时提升令牌效率和延迟性能。具体地，利用快速系统1计算给定查询的答案熵，该分数用于评估样本的缩放潜力，从而在系统2中实现动态自一致性。得益于系统1提供的提前精准估计，所提方法在保持低延迟的同时显著减少令牌使用。实验表明，该方法相比现有技术可实现最高47%的令牌消耗减少和43%的推理延迟降低，且性能损失不显著。
原文摘要 · Abstract (English)
Test-time scaling improves the inference performance of Large Language Models (LLMs) but also incurs substantial computational costs. Although recent studies have reduced token consumption through dynamic self-consistency, they remain constrained by the high latency of sequential requests. In this paper, we propose SeerSC, a dynamic self-consistency framework that simultaneously improves token efficiency and latency by integrating System 1 and System 2 reasoning. Specifically, we utilize the rapid System 1 to compute the answer entropy for given queries. This score is then used to evaluate the potential of samples for scaling, enabling dynamic self-consistency under System 2. Benefiting from the advance and accurate estimation provided by System 1, the proposed method can reduce token usage while simultaneously achieving a significant decrease in latency through parallel generation. It outperforms existing methods, achieving up to a 47% reduction in token consumption and a 43% reduction in inference latency without significant performance loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。