arXiv:2605.24340cs.LG2026-05

用可学习多项式层提升模型样本效率与鲁棒性,适配多领域任务。

ChainzRule: Sample-Efficient, Robust Deep Learning Across Tabular, NLP, and Vision Tasks

  • 以微分正则化约束梯度,使网络输出低频稳定表示。
  • 少样本下性能超越传统模型,如5%数据达SST-5最优结果。
  • 适合对可解释性与部署可靠性要求高的工业级应用。

跨企业领域的生产级深度学习受制于标注数据昂贵、推理预算紧张及模型不可解释等问题,而学术基准常忽略这些限制。本文提出ChainzRule(CR),一种以可学习多项式层替代常规激活函数的神经架构,通过在前向传播中解析计算层间雅可比惩罚(DREG)实现结构稳定性控制。核心思想是:限制中间导数可促使网络生成低频、结构稳定的表征,从而降低对标注数据量的依赖,提升对分布偏移的鲁棒性,并提供可量化、基于梯度的模型行为分析工具。在五个领域验证:Pima糖尿病分类达85.71%±2.01%,优于支持向量机与XGBoost;SST-5情感分类(冻结编码器)达46.20%±0.37%,仅用5%训练数据即胜过RNTN;BERT微调时达55.79%(对比基线54.9%);Yelp全集序回归使用320万参数,优于10模型平均的66.35%;CIFAR-10-C平均噪声精度提升2.32%。所有结果经邦弗朗尼校正后p值均低于0.05。CR在各数据比例下保持梯度尾部比率τ(p99/均值)为1.01–1.02,显著低于基线的1.07–1.09,我们将其视为样本高效性的机制根源与部署可靠性代理指标。

原文摘要 · Abstract (English)

Production deep learning systems across enterprise domains operate under constraints that academic benchmarks routinely obscure: labeled data is expensive, inference budgets are tight, and models that cannot explain their behavior are difficult to trust and maintain. We present ChainzRule (CR), a neural architecture replacing typical activations with learnable polynomial layers governed by Differential Regularization (DREG), a layer-wise Jacobian penalty computed analytically during the forward pass at standard inference cost. The core claim is that bounding intermediate derivatives forces the network toward low-frequency, structurally stable representations, simultaneously reducing dependence on labeled data volume, improving robustness to distribution shift, and providing a measurable, gradient-based handle on model behavior. Evaluated across five domains, CR achieves $85.71\% \pm 2.01\%$ on Pima Diabetes (statistically superior to SVM and XGBoost), $46.20\% \pm 0.37\%$ on SST-5 sentiment classification with a frozen encoder (superior to RNTN using approximately 5\% of its training data), $55.79\%$ on SST-5 with a fine-tuned BERT backbone (versus BERT-base linear head at $54.9\%$), $70.17\%$ on Yelp Full ordinal regression with 3.2M parameters versus a 10-model average of $66.35\%$, and $+2.32\%$ mean corruption accuracy on CIFAR-10-C. All results with reported $p$-values fall below the $α= 0.05$ threshold after Bonferroni correction. CR maintains a gradient tail ratio $τ$ (p99/mean) of $1.01$--$1.02$ against $1.07$--$1.09$ for all typical activation function baselines across every data fraction, a structural invariant we propose as the mechanistic driver of sample efficiency and a deployment-time proxy for model reliability.

深度学习样本效率可解释性鲁棒性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。