用概率与熵的协同校准，精准识别需重点训练的词汇。
Probability-Entropy Calibration: An Elastic Indicator for Adaptive Fine-tuning
- 引入相对排名指标，融合真实概率与预测熵
- 在数学推理等任务上显著提升性能，优于单一指标方法
- 适合需要精细微调的语言模型优化场景
词级别重加权是一种简单而有效的监督微调控制机制，但现有指标多为单维：真实概率反映下游对齐程度，词元熵反映预训练先验带来的内在不确定性。忽略熵会导致将噪声或易替换词误判为学习关键项，忽略概率则无法体现目标特定对齐。RankTuner提出概率-熵校准信号——相对排名指标，比较真实词元排名与其在预测分布下的期望排名。反向指标作为词级相对尺度，用于重加权微调目标，聚焦真正未充分学习的词元，避免对固有不确定位置过度惩罚。多个骨干模型的实验表明，该方法在数学推理基准上表现一致提升，跨分布推理任务中实现迁移增益，代码生成性能优于仅基于概率或熵的基线方法。
原文摘要 · Abstract (English)
Token-level reweighting is a simple yet effective mechanism for controlling supervised fine-tuning, but common indicators are largely one-dimensional: the ground-truth probability reflects downstream alignment, while token entropy reflects intrinsic uncertainty induced by the pre-training prior. Ignoring entropy can misidentify noisy or easily replaceable tokens as learning-critical, while ignoring probability fails to reflect target-specific alignment. RankTuner introduces a probability--entropy calibration signal, the Relative Rank Indicator, which compares the rank of the ground-truth token with its expected rank under the prediction distribution. The inverse indicator is used as a token-wise Relative Scale to reweight the fine-tuning objective, focusing updates on truly under-learned tokens without over-penalizing intrinsically uncertain positions. Experiments on multiple backbones show consistent improvements on mathematical reasoning benchmarks, transfer gains on out-of-distribution reasoning, and pre code generation performance over probability-only or entropy-only reweighting baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。