PETS通过优化轨迹分配,用更少样本实现更高自一致性。
PETS: A Principled Framework Towards Optimal Trajectory Allocation for Efficient Test-Time Self-Consistency
- 将推理轨迹视为工人,用众包理论设计高效分配策略。
- 在GPQA上达完美自一致性,采样量比均匀分配减少75%(离线)和55%(在线)。
- 适合追求测试时效率的大型模型应用,尤其对预算敏感场景。
测试时扩展可通过聚合随机推理轨迹提升模型性能,但在有限预算下实现高效的测试时自一致性仍是未解难题。本文提出PETS(原则性且高效的测试时自一致性),首次从优化框架出发系统研究轨迹分配问题。核心是引入自一致性率——与无限预算多数投票一致性的度量,使采样分配具备理论基础并可严格分析。在离线场景中,将推理轨迹建模为工人,借鉴经典众包理论,获得理论保证并设计出基于多数投票的高效分配算法。在在线流式场景中,提出受离线框架启发的新方法,根据问题难度动态调整预算,同时保持强理论保障与计算高效。实验表明,PETS始终优于均匀分配。在GPQA数据集上,两种设置均实现完美自一致性,采样预算相比均匀分配降低最高达75%(离线)和55%(在线)。代码已开源:https://github.com/ZDCSlab/PETS。
原文摘要 · Abstract (English)
Test-time scaling can improve model performance by aggregating stochastic reasoning trajectories. However, achieving sample-efficient test-time self-consistency under a limited budget remains an open challenge. We introduce PETS (Principled and Efficient Test-TimeSelf-Consistency), which initiates a principled study of trajectory allocation through an optimization framework. Central to our approach is the self-consistency rate, a new measure defined as agreement with the infinite-budget majority vote. This formulation makes sample-efficient test-time allocation theoretically grounded and amenable to rigorous analysis. We study both offline and online settings. In the offline regime, where all questions are known in advance, we connect trajectory allocation to crowdsourcing, a classic and well-developed area, by modeling reasoning traces as workers. This perspective allows us to leverage rich existing theory, yielding theoretical guarantees and an efficient majority-voting-based allocation algorithm. In the online streaming regime, where questions arrive sequentially and allocations must be made on the fly, we propose a novel method inspired by the offline framework. Our approach adapts budgets to question difficulty while preserving strong theoretical guarantees and computational efficiency. Experiments show that PETS consistently outperforms uniform allocation. On GPQA, PETS achieves perfect self-consistency in both settings while reducing the sampling budget by up to 75% (offline) and 55% (online) relative to uniform allocation. Code is available at https://github.com/ZDCSlab/PETS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。