arXiv:2510.20653stat.MLcs.AI2025-10被引 2

对比推理时反思与预算调节,找到质量、成本、速度的最优平衡点。

Finding the Sweet Spot: Trading Quality, Cost, and Speed During Inference-Time LLM Reflection

  • 通过多轮反思和计算预算调节提升模型性能。
  • 数学推理任务中效果提升最高达220%。
  • 适合需权衡资源与性能的落地应用开发者。

随着大语言模型持续演进,从业者在不重新训练模型的前提下,面临增强推理性能的多种选择,包括预算调节和多步技术如自我反思。尽管这些方法能提升输出质量,却在准确率、成本和延迟之间制造复杂权衡,且不同领域表现差异显著。本文系统比较了自反性与预算调节在数学推理和翻译任务中的表现。评估了Anthropic Claude、Amazon Nova及Mistral系列等主流模型,在不同反思深度与计算预算下的性能,构建出帕累托最优性能边界。分析显示,自反性在数学推理任务中效果提升高达220%,且其成效受反思轮次深度与反馈机制质量影响显著。为验证实际价值,我们在Zalando旗下Lounge部署了增强型自反式营销内容本地化系统,结果表明其效果具有市场依赖性,强调了特定领域评估的重要性。研究为在不同场景下选择最优推理策略提供了可操作指导。代码已开源:https://github.com/aws-samples/sample-genai-reflection-for-bedrock。

原文摘要 · Abstract (English)

As Large Language Models (LLMs) continue to evolve, practitioners face increasing options for enhancing inference-time performance without model retraining, including budget tuning and multi-step techniques like self-reflection. While these methods improve output quality, they create complex trade-offs among accuracy, cost, and latency that remain poorly understood across different domains. This paper systematically compares self-reflection and budget tuning across mathematical reasoning and translation tasks. We evaluate prominent LLMs, including Anthropic Claude, Amazon Nova, and Mistral families, along with other models under varying reflection depths and compute budgets to derive Pareto optimal performance frontiers. Our analysis reveals substantial domain dependent variation in self-reflection effectiveness, with performance gains up to 220\% in mathematical reasoning. We further investigate how reflection round depth and feedback mechanism quality influence performance across model families. To validate our findings in a real-world setting, we deploy a self-reflection enhanced marketing content localisation system at Lounge by Zalando, where it shows market-dependent effectiveness, reinforcing the importance of domain specific evaluation when deploying these techniques. Our results provide actionable guidance for selecting optimal inference strategies given specific domains and resource constraints. We open source our self-reflection implementation for reproducibility at https://github.com/aws-samples/sample-genai-reflection-for-bedrock.

大模型推理自反思性能优化资源权衡

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。