arXiv:2507.22767cs.LGcs.AI2025-07中稿 · GECCO 2026

让神经网络老师更平滑,能教出更准的可读公式

Teaching the Teacher: The Role of Teacher-Student Smoothness Alignment in Genetic Programming-based Symbolic Distillation

  • 用雅可比和利普希茨惩罚约束教师模型平滑性
  • 在20个数据集上,学生模型R²提升显著且统计显著
  • 适合追求可解释性的机器学习研究者

通过遗传编程实现深度神经网络的符号化蒸馏,是可解释人工智能的有前景路径;但标准流程常导致符号模型预测精度低。我们发现功能复杂度不匹配是主因:标准神经网络学得函数不规则,而符号回归偏好简洁,导致学生模型无法充分学习。为此,我们提出框架,在训练教师时引入雅可比与利普希茨正则化,增强其函数平滑性,以提升学生模型表现。通过20个数据集、50次独立实验,验证了平滑性对齐的有效性:经平滑教师蒸馏的学生模型在R²上显著优于标准管道。消融实验表明该策略对不同学生算法均有效,证明教师-学生平滑性对齐是符号蒸馏的关键。

原文摘要 · Abstract (English)

Obtaining human-readable symbolic formulas via genetic programming-based symbolic distillation of a deep neural network trained on the target dataset presents a promising yet underexplored path towards explainable artificial intelligence (XAI); however, the standard pipeline frequently yields symbolic models with poor predictive accuracy. We identify a fundamental misalignment in functional complexity as the primary barrier to achieving better accuracy: standard Artificial Neural Networks (ANNs) often learn accurate but highly irregular functions, while Symbolic Regression typically prioritizes parsimony, often resulting in a much simpler class of models that are unable to sufficiently distill or learn from the ANN teacher. To bridge this gap, we propose a framework that actively regularizes the teacher's functional smoothness using Jacobian and Lipschitz penalties, aiming to distill better student models than the standard pipeline. We characterize the trade-off between predictive accuracy and functional complexity through a robust study involving 20 datasets and 50 independent trials. Our results demonstrate that students distilled from smoothness-regularized teachers achieve statistically significant improvements in R^2 scores, compared to the standard pipeline. We also perform ablation studies on the student model algorithm. Our findings suggest that smoothness alignment between teacher and student models is a critical factor for symbolic distillation.

符号回归可解释AI蒸馏平滑性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。