提出可微零一损失新方法，提升大批次训练下的分类泛化能力。
Differentiable Zero-One Loss via Hypersimplex Projections
- 通过超单纯形投影构建平滑的可微近似，实现零一损失的端到端优化。
- 在大批次训练中显著提升泛化性能，缩小传统大批次训练的性能差距。
- 适用于需要高精度分类的场景，尤其适合对梯度敏感的模型设计。
近年来机器学习强调将结构化优化模块融入端到端可微模型，以增强归纳偏置并更好对齐任务目标。本文提出一种新的零一损失可微近似，该损失长期被视为分类性能的黄金标准，但因其不可微而难以用于梯度优化。我们通过约束优化框架，在n,k维超单纯形上构造了保持顺序的光滑投影，引入名为Soft-Binary-Argmax的新算子。推导其数学性质后，展示了如何高效计算其雅可比矩阵，并集成至二分类与多分类学习系统中。实验证明，该方法通过在输出logits上施加几何一致性约束，在大批次训练下显著提升泛化能力，有效缩小了传统大批次训练中的性能差距。
原文摘要 · Abstract (English)
Recent advances in machine learning have emphasized the integration of structured optimization components into end-to-end differentiable models, enabling richer inductive biases and tighter alignment with task-specific objectives. In this work, we introduce a novel differentiable approximation to the zero-one loss-long considered the gold standard for classification performance, yet incompatible with gradient-based optimization due to its non-differentiability. Our method constructs a smooth, order-preserving projection onto the n,k-dimensional hypersimplex through a constrained optimization framework, leading to a new operator we term Soft-Binary-Argmax. After deriving its mathematical properties, we show how its Jacobian can be efficiently computed and integrated into binary and multiclass learning systems. Empirically, our approach achieves significant improvements in generalization under large-batch training by imposing geometric consistency constraints on the output logits, thereby narrowing the performance gap traditionally observed in large-batch training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。