用可测试的对冲能力解释模型泛化，揭示学习者如何权衡数据拟合与样本依赖。
Bounded-Rationality, Hedging, and Generalization
- 将学习过程建模为有限理性决策，通过输入到输出的信道变化衡量学习行为
- 提出上下界曲线，分别表征训练损失与样本依赖的最低代价和上界证书
- 无需模型结构信息，仅凭对缩放损失和局部扰动的响应即可恢复对冲能力
学习者不仅拟合数据，还决定训练样本对其输出的影响强度以及能容忍的偏差程度。本文将这一关系视为一个有限理性决策问题，其基本对象是样本到输出的诱导信道。学习者的响应规律决定了该信道中哪些变化是廉价或昂贵的，从而产生训练损失与样本依赖之间的下界权衡曲线，以及匹配的上界证书曲线。当响应规律由f-散度正则项表示时，这些曲线位于正则项的天然信息几何中，其中KL散度对应香农互信息。我们展示了如何通过观察学习者对缩放损失和局部损失扰动的响应，从黑箱行为中恢复出对冲能力和两条曲线。在学习中，总体损失等于经验损失加上特定训练样本引起的畸变。若恢复出的对冲能力能覆盖此畸变，则提供了一个可验证的泛化证书。因此，泛化被视作学习者自身响应规律可测试的对冲性质。
原文摘要 · Abstract (English)
A learner does not only fit data; it also determines how strongly the training sample may shape its output and how much distortion it can hedge. We study this relation as a bounded-rational decision problem whose primitive object is the induced channel from samples to outputs. The learner's response law determines which changes in this channel are cheap or costly, and therefore induces both a lower tradeoff curve between training loss and sample dependence and a matched upper certificate curve. When the response law is represented by an $f$-divergence regularizer, these curves live in the regularizer's native information geometry, with KL as the special case corresponding to Shannon mutual information. We show how the hedge and the two curves can be recovered from black-box behavior by observing responses to scaled losses and local loss perturbations. In learning, population loss is empirical loss plus the distortion induced by the particular training sample. The recovered hedge gives a practical certificate when it covers that distortion. Thus generalization is treated as a testable hedging property of the learner's own response law.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。