arXiv:2605.22940cs.LGcs.AI2026-05被引 1

提出新型动态学习框架,用熵调控提升模型在真实环境中的稳定性和可控性。

Human-Centered Learning Mechanics: A Dynamical Framework for Entropy-Regulated Representation Learning

论文配图:Human-Centered Learning Mechanics: A Dynamical Framework for Entropy-Regulated Representation Learning
图 1 · 摘自论文原文
  • 引入有效熵概念,识别并避免熵正则化失效的退化状态。
  • 证明几何熵代理(如对数行列式协方差)能产生更强更稳的信息力。
  • 适合关注模型鲁棒性、人机协同与开放系统学习的研究者。

深度学习被视为参数空间中的动态过程,但现有理论多将其视为封闭优化系统,难以应对真实世界中不确定性、资源限制、分布漂移、下游决策风险和人类反馈等挑战。本文提出人类中心学习力学(HCLM),一个动态且基于信息论的开放可控学习框架。核心思想是:熵正则化仅在所选熵代理能沿优化轨迹生成非退化的信息力时才有效;否则熵项可能产生弱、不稳定或错位梯度,导致动态退化为普通损失最小化。我们引入有效熵概念,研究可计算的几何熵代理,包括基于方差和对数行列式协方差的近似方法。论文有三项贡献:第一,通过有效信息力形式化熵正则化,并刻画退化熵区间;第二,在明确假设下推导收敛性、熵流、Wasserstein梯度流及噪声表示泛化结果;第三,将类似缩放律的行为解释为信息注入、熵耗散与残余风险之间的平衡,而非无条件推导经验缩放律。受控表示学习实验支持该假设:几何熵代理,尤其是对数行列式协方差熵,比软最大归一化熵能诱导更强更稳定的信 息力。

原文摘要 · Abstract (English)

Deep learning is increasingly viewed as a dynamical process in parameter space, yet many existing theories still treat training as a closed optimization system. This view is limited for real-world AI, where models operate under uncertainty, resource constraints, distribution shift, downstream decision risks, and human feedback. We propose Human-Centered Learning Mechanics (HCLM), a dynamical and information-theoretic framework for open and controlled learning systems. The central idea is that entropy regularization is useful only when the chosen entropy surrogate generates a non-degenerate information force along the optimization trajectory. Otherwise, entropy terms may produce weak, unstable, or misaligned gradients, causing the dynamics to collapse toward ordinary loss minimization. We introduce the notion of effective entropy and study tractable geometric entropy surrogates, including variance-based and log-determinant covariance proxies. The paper makes three contributions. First, it formalizes entropy regularization through effective information force and characterizes degenerate entropy regimes. Second, it derives convergence, entropy-flow, Wasserstein-gradient-flow, and noisy-representation generalization results under explicit assumptions. Third, it offers a conditional dynamical interpretation of scaling-law-like behavior as a balance between information injection, entropy dissipation, and residual risk, without claiming an unconditional derivation of empirical neural scaling laws. Controlled representation-learning experiments support the hypothesis that geometric entropy surrogates, especially log-determinant covariance entropy, induce stronger and more stable information forces than softmax-normalized entropy.

动态学习熵正则化信息力可控学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。