用令牌优先机制提升微调效果，让模型生成更符合人类需求。
Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority
- 将微调视为分布重塑，通过令牌优先精准对齐数据与目标
- 发现正向优先可过滤噪声，符号优先能消除有害内容
- 适合关注模型对齐与生成质量优化的研究者
从拟合经验数据到实现真正的人类效用，其根本瓶颈在于粒度不匹配：细粒度的自回归生成常由粗粒度或均匀信号监督。本文主张令牌优先（Token Priority）是关键桥梁，将监督微调（SFT）重新定义为精确的分布重塑过程，使原始数据与理想对齐流形对齐。我们以这一统一视角分析近期突破，将其分为两类：正向优先用于噪声过滤，符号优先用于消除有害模式。我们重新审视现有进展与局限，识别关键挑战，并提出未来研究方向。
原文摘要 · Abstract (English)
The transition from fitting empirical data to achieving true human utility is fundamentally constrained by a granularity mismatch, where fine-grained autoregressive generation is often supervised by coarse or uniform signals. This position paper advocates Token Priority as the essential bridge, formalizing Supervised Fine-Tuning (SFT) not as simple optimization but as a precise distribution reshaping process that aligns raw data with the ideal alignment manifold. We analyze recent breakthroughs through this unified lens, categorizing them into two distinct regimes: Positive Priority for noise filtration and Signed Priority for toxic modes unlearning. We revisit existing progress and limitations, identify key challenges, and suggest directions for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。