用强化学习让模型分步改进预测，自动调节计算量。
Do Not Imitate, Reinforce: Iterative Classification via Belief Refinement
- 模型分步迭代更新分类概率，不再一次定论
- 在图像分类上准确率相当，校准度更好
- 适合需要自适应计算的实时场景
标准监督分类让模型盲目模仿完美标注，单次完成，无法根据输入复杂度调整计算。同时训练目标迫使模型对训练数据表现绝对自信，导致评估时过度自信。我们提出强化迭代分类（RIC），用强化学习替代模仿式训练。RIC采用循环智能体，分步更新类别预测分布，通过预测质量提升获得奖励。价值函数自然提供停止条件，估计改进余地。理论证明，该方法与交叉熵最优预测一致，并实现任意时间分类。在图像分类基准上，RIC达到与监督基线相当的准确率，校准性更优，且能自适应分配计算资源。
原文摘要 · Abstract (English)
Standard supervised classification trains models to imitate the exact labels provided by a perfect oracle. This imitation happens in a single pass, restricting the model to a fixed compute budget even when inputs vary in complexity. Moreover, the rigid training objective forces the model to express absolute certainty on its training data, resulting in overconfident predictions during evaluation. We propose Reinforced Iterative Classification (RIC), which replaces the imitative objective with Reinforcement Learning (RL). RIC deploys a recurrent agent that iteratively updates a predictive distribution over classes, receiving reward for stepwise improvement in prediction quality. The value function provides a natural halting criterion by estimating the remaining scope for improvement. We prove that the iterative formulation recovers the same optimal predictions as cross-entropy while yielding an anytime classifier. On image classification benchmarks, RIC matches the accuracy of supervised baselines with improved calibration and learns to allocate computation adaptively across inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。