发现损失函数变化会引发新迁移失效模式，影响模型表现。
Loss-Shift Transfer via Bayes Quotients
- 用贝叶斯商数构建损失排序机制，揭示损失变化的深层影响。
- 在固定数据分布下，不同损失导致最优表示差异，表现差距可量化。
- 适用于理解模型迁移中损失敏感问题的研究者或工程师。
迁移学习通常被视作分布偏移的结果。本文识别出一种正交的失败模式：数据分布固定，但损失函数发生变化，这种情形称为“损失偏移”（loss shift）。一个损失函数决定了输入X中哪些信息对预测目标Y是贝叶斯相关的，因此即使联合分布P(X,Y)相同，两个不同的损失也可能需要不同的表示。本文利用贝叶斯商数对损失进行排序，实现损失间的精炼性比较。在贝叶斯商数框架下，严格精炼关系带来直接的定性障碍：对于更粗略损失的源端最小表示，无法满足严格更精细的目标损失。对于有限输出的对数损失，该障碍转化为精确的定量恒等式：额外风险等于表示中被丢弃的关于Y的条件信息。在受控环境、学习表示、合成图像和真实图像等多种设置下的实验验证了该理论预测——在相同数据分布下，分类性能相当的表示，在对数损失下的最优表现仍可能不同。
原文摘要 · Abstract (English)
Transfer learning is usually studied as a consequence of distribution shift. This paper identifies an orthogonal failure mode in which the data distribution is fixed and the loss changes. This setting is called \emph{loss shift}. A loss determines which information in \(X\) is Bayes-relevant, and two losses may therefore require different representations even under the same joint law \(P(X,Y)\). The idea is formalized using Bayes quotients, which allow losses to be ordered by refinement. In the Bayes-quotient formulation, strict refinement gives an immediate qualitative obstruction. A source-minimal representation for a coarser loss is insufficient for a strictly finer target loss. For finite-output log loss, this obstruction becomes an exact quantitative identity. The excess risk is the conditional information about \(Y\) discarded by the representation. Experiments in controlled, learned, synthetic-image, and real-image settings show the predicted effect, i.e., classification-equivalent representations can have different optimal log-loss performance under a fixed data distribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。