通过智能剔除低损失样本，实现高效训练且不牺牲精度。
K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)
- 基于误差筛选低损失样本，仅对高损失样本计算梯度，降低计算开销。
- 在28.4%计算节省下，测试AUC达0.9991，接近全批量训练效果。
- 适用于资源受限场景，尤其适合处理数据不平衡或噪声数据的模型训练。
本文提出K-ABENA（K-自适应反向传播与基于误差的N排除算法），一种选择性梯度计算框架，通过在反向传播中排除部分低损失（“次要”）样本，降低每轮训练的计算成本。其标准形式（v3）结合防御性混合采样与霍维茨-汤普森逆概率重加权，得到无偏的霍维茨-汤普森梯度估计器（引理2），其自归一化变体偏差为O(1/m)并具明确常数（引理3）。我们证明了在该估计器下，SGD具有O(1/√T)的非凸收敛性，附加项量化残余偏差（定理1）。进一步证明：未补偿的基于损失的选择（包括OHEM、SBP及早期K-ABENA版本）在选择偏差有界处无法收敛至任何极小值点（命题2）。实验显示，在0.17%类别不平衡下，未补偿方法测试AUC仅为0.53–0.62，而全批量SGD为0.9998；补偿估计器在相同28.4%计算节省下达到0.9991。在真实数据集（乳腺癌、数字、葡萄酒、糖尿病）上，补偿估计器与全批量训练无统计差异（配对置换检验，p ≥ 0.5），同时节省28–54%每轮梯度计算。保留一种带偏的“正则化模式”（早期半域版本），其偏置可精确分解（引理5），但存在明确局限：在40%标签噪声下准确率降至0.386（基线0.832），极端不平衡下AUC降为0.53。本文所有优势与限制均经证明或实测，所有实验均为CPU级（NumPy/scikit-learn），范围明确。
原文摘要 · Abstract (English)
We present K-ABENA (K-Adaptive Backpropagation with Error-based N-exclusion Algorithm), a selective gradient computation framework that reduces per-iteration training cost by excluding a fraction of low-loss ("minor") observations from the backward pass. Its canonical form (v3) combines a defensive-mixture sampling design over the minor set with Horvitz-Thompson inverse-probability reweighting, yielding a design-unbiased Horvitz-Thompson gradient estimator (Lemma 2) and whose self-normalized practical variant carries a bias of order O(1/m) with an explicit constant (Lemma 3). We prove an O(1/sqrt(T)) non-convex convergence guarantee for SGD under the estimator, with an additive term that quantifies the residual bias (Theorem 1). We further prove that uncompensated loss-based selection - a family that includes OHEM, SBP, and the two earlier K-ABENA variants - admits no stationary point at any minimizer where its selection bias is bounded away from zero (Proposition 2), and we quantify this failure empirically: at 0.17% class imbalance, uncompensated variants reach test AUC 0.53-0.62 versus 0.9998 for full-batch SGD, while the compensated estimator attains 0.9991 at identical 28.4% compute savings. On real datasets (Breast Cancer, Digits, Wine, Diabetes) the compensated estimator is statistically indistinguishable from full-batch SGD (paired permutation tests, p >= 0.5; Section 7) while saving 28-54% of per-epoch gradient computation. A biased "regularized mode" (the earlier half-domain variant) is retained as an option with a proven exact bias decomposition (Lemma 5) and quantified contraindications: it collapses to 0.386 accuracy under 40% label noise (baseline: 0.832) and to 0.53 AUC under extreme imbalance. Every advantage and every limitation reported in this paper is either proved or measured; all experiments are CPU-scale (NumPy/scikit-learn) and their scope is stated explicitly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。