AdaCap让小数据神经网络更稳定，提升残差模型表现
AdaCap: An Adaptive Contrastive Approach for Small-Data Neural Networks
- 用排列对比损失+闭式映射，自适应增强小数据训练
- 85个真实数据集上显著优于传统方法，残差模型提升最明显
- 可预测何时适用，适合小样本建模场景
神经网络在小规模表格数据上表现不佳，树模型仍占主导。我们提出自适应对比方法（AdaCap），结合基于排列的对比损失与基于Tikhonov的闭式输出映射。在85个真实世界回归数据集和多种架构上，AdaCap在小样本条件下均带来一致且统计显著的性能提升，尤其对残差模型效果突出。基于数据特征（大小、偏度、噪声）训练的元预测器能准确判断AdaCap是否有效。结果表明，AdaCap是一种精准的正则化机制，强化了神经网络最脆弱的环节。所有结果与代码已公开于https://github.com/BrunoBelucci/adacap。
原文摘要 · Abstract (English)
Neural networks struggle on small tabular datasets, where tree-based models remain dominant. We introduce Adaptive Contrastive Approach (AdaCap), a training scheme that combines a permutation-based contrastive loss with a Tikhonov-based closed-form output mapping. Across 85 real-world regression datasets and multiple architectures, AdaCap yields consistent and statistically significant improvements in the small-sample regime, particularly for residual models. A meta-predictor trained on dataset characteristics (size, skewness, noise) accurately anticipates when AdaCap is beneficial. These results show that AdaCap acts as a targeted regularization mechanism, strengthening neural networks precisely where they are most fragile. All results and code are publicly available at https://github.com/BrunoBelucci/adacap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。