提出四维会计框架,量化学习设备的记忆、适配与价值转化效率。
Thermodynamics of Learning: A Typed Four-Component Accounting of Memory, Fit, and Value
- 构建记忆、拟合、搜索与资本价值四元分离模型,区分记录与未来价值。
- 证明在特定条件下,更新不提升价值,且资本化效率上限为1。
- 揭示价值保留机制,适用于任务分布变化下的有限设备评估。
有限学习设备所记录的内容与其未来任务中的价值并非同一概念。本文为有限状态学习设备建立类型化会计体系,分离出四个组成部分:训练侧拟合函数Φ_{fit}、记录相关性存量J_D=I(M;D)、更新侧搜索账本σ_M,以及操作资本价值V(M;T,b)。该价值定义为知情协议类与删除记忆读取端口后重优化的盲类之间的功耗差距。首先,对任意n,存在一类设备使记录相关性与世界相关性各增长n ln 2,但资本收益恰好为零;在flat^*模式下,无数据更新永远不增加V。其次,给出flat^*提取恒等式与通用账本恒等式,在满足(F5')稳定性及无丢弃记录相关性条件时,资本化效率η_cap = ΔV/(kT σ_M) 的上界为1,并明确等号成立的充要条件。第三,针对保留差距L_gen与保留率ρ_gen(前者无符号限制,后者仅对正训练价值定义且不限于[0,1]),建立双层对齐域:在无需记录侧信息独立性的前提下,精确给出价值与侧信息调整后的记录拟合量I(M';D|Y)之间的交换率;在联合侧信息中立条件(M,D)⊥Y下,给出原始记录存量交换率,其边界由显式的一次性密码见证标记。这些结论均关于任务分布漂移下有限设备的价值保留,而非统计泛化理论。
原文摘要 · Abstract (English)
What a finite learning device has recorded and what will hold value for it on future tasks are not the same quantity. We develop a typed accounting for finite-state learning devices that separates four components: a training-side fit functional $Φ_{\mathrm{fit}}$, the record-correlation stock $J_{D}=I(M;D)$, an update-side search ledger $σ_{M}$, and an operational capital value $V(M;T,b)$. This value is the work gap between an informed protocol class and a blind class obtained by deleting the memory-read port and re-optimizing from scratch. (I) Separation: for every $n$, there is a device family on which record correlation and world correlation grow by $n\ln 2$ while the capital gain is exactly zero. In the $\mathrm{flat}^{*}$ regime, data-free updates never increase $V$. (II) Capitalization ledger: an exact $\mathrm{flat}^{*}$ extraction identity and a universal ledger identity give, for (F5$'$)-stable $M$-local updates under a no-discarded-record-correlation condition (f), the bound $η_{\mathrm{cap}}\le 1$ for the capitalization efficiency $η_{\mathrm{cap}}=ΔV/(k T\,σ_{M})$, together with necessary and sufficient conditions for equality. (III) Value retention: for the retention gap $L_{\mathrm{gen}}$ and retention ratio $ρ_{\mathrm{gen}}$ (the former carries no sign constraint; the latter is defined for positive training-side value and is not confined to $[0,1]$) we give a two-layer alignment domain: an exact exchange rate between value and the side-information-adjusted record fit $I(M';D\mid Y)$ without any record-side-information independence assumption, and a raw record-stock exchange rate under a joint side-information neutrality condition $(M,D)\perp Y$, whose boundary is marked by an explicit one-time-pad witness. These are statements about finite-device value retention under task-distribution shift, not a theory of statistical generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。