arXiv:2510.18905cs.LGcs.AI2025-10

提出三维优化框架,同时平衡模型推理的精度、成本与延迟。

3D Optimization for AI Inference Scaling: Balancing Accuracy, Cost, and Latency

  • 构建统一决策空间,联合优化精度、成本与延迟三要素。
  • 小模型配最优推理策略,性能可超大型模型且成本更低。
  • 基于帕累托前沿的拐点优化效果最佳,适配不同部署环境。

AI推理扩展常依赖一维启发式方法(固定推理轮次)或二维权衡(如精度与计算量),忽略成本与延迟约束。本文提出三维优化框架,在统一决策空间中协同校准精度、成本与延迟,实现约束感知的推理扩展。通过在三个典型场景和九个模拟的大语言模型上进行蒙特卡洛仿真,评估四种优化方法应对三维多目标优化问题的表现。将推理扩展置于多目标优化框架下,生成1D与2D方法无法捕捉的可行解空间,支持环境自适应选择推理缩放系数 $k$。结果表明,基于帕累托前沿的拐点优化取得最佳平衡,而当精度优先时,精度最大化策略仍具优势。进一步显示,小模型结合最优推理缩放后,性能可媲美甚至超越大模型,成本仅为一小部分。该框架为多样化运行条件下部署感知的推理扩展提供了理论基础。

原文摘要 · Abstract (English)

AI inference scaling is often tuned through 1D heuristics (a fixed reasoning pass) or 2D bivariate trade-offs (e.g., accuracy vs. compute), which fail to consider cost and latency constraints. We introduce a 3D optimization framework that jointly calibrates accuracy, cost, and latency within a unified decision space, enabling constraints-aware inference scaling. Using Monte Carlo simulations across three representative scenarios and nine simulated large language models, we evaluate four optimization methods to address the 3D multi-objective optimization (MOO) problem. Framing inference scaling in MOO shapes a feasible space that 1D and 2D optimizations fail to capture, enabling environment-adaptive selection of the inference scaling~$k$. Results show that knee-point optimization based on Pareto frontiers achieves the best balance, while accuracy-maximization remains favorable when accuracy is prioritized. Our results further show that smaller models, when combined with optimal inference scaling, can match or exceed the performance of larger models at a fraction of the cost. The framework establishes a theoretical foundation for deployment-aware inference scaling across diverse operational conditions.

推理优化多目标优化成本控制模型部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。