通过双模型自助采样，训练中实时估计不确定性并优化模型
Twin-Boot: Uncertainty-Aware Optimization via Online Two-Sample Bootstrapping
- 并行训练两个模型，用自助采样数据同步更新
- 通过模型差异衡量局部不确定性，提升泛化与校准效果
- 适合低数据量、过参数化场景下的可靠模型训练
标准梯度下降方法仅生成点估计，缺乏置信度衡量。在过参数化和低数据场景下，模型参数远多于数据量，易过拟合。传统自助法虽可估算不确定性，但直接用于深度学习不切实际：需训练多个副本，估计为事后生成且无法指导学习，且默认各次运行收敛至相似最优解——这在非凸优化中不成立。本文提出双自助梯度下降（Twin-Boot），一种将不确定性估计融入优化的重采样方法。两个相同模型并行训练于独立自助样本，定期均值重置使两轨迹保持在同一谷底，其发散程度反映局部（谷内）不确定性。训练过程中，利用此估计自适应地采样权重，提供正则化，偏好平坦解。在深度神经网络及复杂高维逆问题中，该方法改善了模型校准与泛化性能，生成可解释的不确定性图谱。
原文摘要 · Abstract (English)
Standard gradient descent methods yield point estimates with no measure of confidence. This limitation is acute in overparameterized and low-data regimes, where models have many parameters relative to available data and can easily overfit. Bootstrapping is a classical statistical framework for uncertainty estimation based on resampling, but naively applying it to deep learning is impractical: it requires training many replicas, produces post-hoc estimates that cannot guide learning, and implicitly assumes comparable optima across runs - an assumption that fails in non-convex landscapes. We introduce Twin-Bootstrap Gradient Descent (Twin-Boot), a resampling-based training procedure that integrates uncertainty estimation into optimization. Two identical models are trained in parallel on independent bootstrap samples, and a periodic mean-reset keeps both trajectories in the same basin so that their divergence reflects local (within-basin) uncertainty. During training, we use this estimate to sample weights in an adaptive, data-driven way, providing regularization that favors flatter solutions. In deep neural networks and complex high-dimensional inverse problems, the approach improves calibration and generalization and yields interpretable uncertainty maps.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。