从几何视角揭示随机学习中隐式偏差的成因并实现逆向设计
Understanding and inverse design of implicit bias in stochastic learning: a geometric perspective
- 基于梯度噪声与损失对称性的几何作用，构建隐式偏差理论框架
- 预测新现象并解释已有行为，成功在多种架构上计算出偏差分布
- 可逆向设计参数化以控制偏差，实现稀疏性与谱稀疏性调控
机器学习中的一个核心挑战是理解在过参数化模型中，为何学习动态会选择众多具有相同损失值的解之一——这一现象称为隐式偏差。控制这种偏差可直接影响学习到的表征，而表征正是现代人工智能系统可解释性、鲁棒性和推理能力的核心。然而，尽管其重要性显著，现有解释仍多为零散且缺乏统一机制。本文提出一个理论性且可构造的框架，指出隐式偏差源于梯度噪声与损失函数连续对称性相互作用所引发的几何修正。我们在多种架构上计算了由此产生的偏差，预测了新行为并解释了已知现象。该方法还支持逆向设计：通过设计保持预测性能的参数化方式，可主动塑造偏差，稀疏性与谱稀疏性成为典型实例。数值实验验证了理论，并在受控环境中证实了逆向设计的有效性。
原文摘要 · Abstract (English)
A key challenge in machine learning is to explain how learning dynamics select among the many solutions that achieve identical loss values in overparameterized models - a phenomenon known as implicit bias. Controlling this bias provides a direct mechanism on learned representations, which are central to interpretability, robustness, and reasoning in modern AI systems. Yet, despite its importance, existing explanations remain largely ad hoc and lack a unifying mechanism. We develop a theoretical and constructive framework in which implicit bias emerges as a geometric correction induced by the interplay between gradient noise and continuous symmetries of the loss. We compute the induced bias across a range of architectures, predicting new behaviors and explaining known ones. The approach also enables inverse design: by engineering predictor - preserving parameterizations, it is possible to shape the bias, with sparsity and spectral sparsity emerging as canonical instances. Numerical experiments support the theory and validate the inverse - design framework in controlled settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。