通过软对称性诱导低复杂度解，提升神经网络泛化与鲁棒性。
Softly Induced Functional Simplicity: Implications for Neural Network Generalisation, Robustness, and Distillation
- 引入软对称性先验，使损失函数出现近似简并态
- 低复杂度解在高能物理分类任务中泛化性能更优
- 适用于需要高效压缩与鲁棒性的模型部署场景
从高维输入数据中学习鲁棒且可泛化的抽象是机器学习及其在高能物理（HEP）应用中的核心挑战。已知功能复杂度较低的解能产生更具泛化能力且对输入扰动更鲁棒的抽象。在复杂的假设空间中，归纳偏置通过塑造优化过程中的损失几何结构，使这类解可学习。在一项HEP分类任务中，我们表明，软对称性尊重的归纳偏置会在损失函数中产生近似简并性，我们将其识别为伪戈尔德斯通模式。通过基于一阶原理的海森分析和可压缩性度量来量化功能复杂度。结果表明，低复杂度解生成的抽象具有更强的泛化能力、更高的鲁棒性，并且更易于高效蒸馏。
原文摘要 · Abstract (English)
Learning robust and generalisable abstractions from high-dimensional input data is a central challenge in machine learning and its applications to high-energy physics (HEP). Solutions of lower functional complexity are known to produce abstractions that generalise more effectively and are more robust to input perturbations. In complex hypothesis spaces, inductive biases make such solutions learnable by shaping the loss geometry during optimisation. In a HEP classification task, we show that a soft symmetry respecting inductive bias creates approximate degeneracies in the loss, which we identify as pseudo-Goldstone modes. We quantify functional complexity using metrics derived from first principles Hessian analysis and via compressibility. Our results demonstrate that solutions of lower complexity give rise to abstractions that are more generalisable, robust, and efficiently distillable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。