让大模型推理更快：用一次预计算替代每次迭代优化。
Amortized Latent Steering: Low-Cost Alternative to Test-Time Optimization
- 提前计算成功与失败生成的隐状态差异，推理时直接修正
- 在GSM8K和MATH-500上实现2-5倍加速，效率提升超100%
- 适合需要低延迟推理的生产环境部署
测试时优化因高昂推理成本难以规模化——如迭代精炼和多步验证需比标准解码多出10-100倍计算量。潜空间测试时优化方法（如LatentSeek）虽更直接，但仍需每查询进行多次反向传播的优化循环。我们提出消减式潜空间引导（ALS），将迭代优化压缩为一次离线计算的向量，推理时仅以固定开销应用。ALS计算成功与失败生成中隐状态的均值差，利用该方向校准模型隐表示：当解码偏离成功流形时，自动将其拉回。在GSM8K和MATH-500基准上，ALS相比迭代方法实现2-5倍提速，性能媲美或超越贪婪链式思考（CoT）与自一致性基线，效率-准确率权衡最高提升101%。结果表明，大部分潜空间优化收益可离线捕获，使复杂推理技术适用于实际部署。代码见https://github.com/negbuna/ALS。
原文摘要 · Abstract (English)
Test-time optimization remains impractical at scale due to prohibitive inference costs--techniques like iterative refinement and multi-step verification can require $10-100\times$ more compute per query than standard decoding. Latent space test-time optimization methods like LatentSeek offer a more direct approach by steering hidden representations, but still demand expensive per-query optimization loops with multiple backward passes. We propose Amortized Latent Steering (ALS), which collapses this iterative optimization into a single offline-computed vector applied at constant cost during inference. ALS computes the mean difference between hidden states from successful versus unsuccessful generations, then uses this direction to calibrate the model's hidden representations: when decoding drifts away from the success manifold, ALS nudges activations back toward it. Across GSM8K and MATH-500 benchmarks, ALS achieves $2-5\times$ speedup over iterative methods while matching or surpassing greedy Chain-of-Thought (CoT) and Self-Consistency baselines, yielding up to 101% improvement in efficiency--accuracy trade-off. These results show that much of latent optimization's benefit can be captured offline, making sophisticated reasoning techniques viable for production deployment. Code is available at https://github.com/negbuna/ALS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。