提出预调优风险分解框架，揭示预测性能的理论极限。
A Risk Decomposition Framework for Pre-Hoc Fine-Tuning Prediction
- 将预调优预测建模为信息受限下的随机估计问题
- 发现优化方差有不可逾越的衰减速率下限
- 给出三类任务阶段图谱，指导低成本探查策略
大模型微调成本高昂，预调优性能预测可显著降低成本。然而，该预测的理论极限尚不明确。本文将其建模为信息约束下的随机估计问题，将预测风险分解为两个部分：内在极限（静态数据-模型兼容性）和可减少的优化方差。证明优化方差的衰减速率存在必要下界，表明无论使用何种预测器，不确定性消散速度均有根本限制。基于此动态，推导出预算最优探查原则，并提出可预测性相图，将任务划分为三类：静态充分、动态关键、噪声主导。在合成与真实世界基准上的大量实验验证了这些理论阶段，并展示了探查策略的高效性。
原文摘要 · Abstract (English)
The high cost of fine-tuning LLMs poses a significant economic barrier; pre-hoc performance prediction offers a critical solution to substantially reduce this expense. However, the theoretical limits of pre-hoc performance prediction remain unexplored. We formulate it as a stochastic estimation problem under information constraints, decomposing prediction risk into two components: an intrinsic limit (static data-model compatibility) and a reducible optimization variance. We prove that optimization variance admits a necessary lower bound on its decay rate, implying fundamental constraints on how quickly uncertainty dissipates, regardless of the predictor used. Based on these dynamics, we derive a budget-optimal probing principle and introduce a predictability phase diagram that organizes tasks into three distinct regimes: Static-Sufficient, Dynamic-Critical, and Noise-Dominant. Extensive experiments on synthetic and real-world benchmarks validate these theoretical regimes and demonstrate the efficiency of our probing strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。