让模型学会评估自身表示的可靠性,提升鲁棒性与稳定性。
Beyond Predictive Uncertainty: Reliable Representation Learning with Structural Constraints
- 在表示空间引入不确定性正则化,显式建模表示可靠性。
- 通过结构约束(如稀疏性、依赖关系)减少冗余变化,增强稳定性。
- 适用于多种模型,无需修改架构,适合高可靠性场景使用。
机器学习中的不确定性估计传统上仅关注预测阶段,默认认为学习到的表示是确定且可靠的。本文挑战这一假设,主张将可靠性作为表示本身的首要属性。提出一种基于结构约束的可靠表示学习框架,显式建模表示层面的不确定性,并利用结构先验(如稀疏性、关系结构、特征组依赖)作为归纳偏置,规范可行表示空间。该方法在表示空间中引入不确定性感知正则化,使表示不仅具备预测能力,还具有稳定性、校准性及对噪声和结构扰动的鲁棒性。结构约束不依赖于完全正确或无噪声的结构,可灵活融入各类表示学习方法,且与具体模型架构无关。
原文摘要 · Abstract (English)
Uncertainty estimation in machine learning has traditionally focused on the prediction stage, aiming to quantify confidence in model outputs while treating learned representations as deterministic and reliable by default. In this work, we challenge this implicit assumption and argue that reliability should be regarded as a first-class property of learned representations themselves. We propose a principled framework for reliable representation learning that explicitly models representation-level uncertainty and leverages structural constraints as inductive biases to regularize the space of feasible representations. Our approach introduces uncertainty-aware regularization directly in the representation space, encouraging representations that are not only predictive but also stable, well-calibrated, and robust to noise and structural perturbations. Structural constraints, such as sparsity, relational structure, or feature-group dependencies, are incorporated to define meaningful geometry and reduce spurious variability in learned representations, without assuming fully correct or noise-free structure. Importantly, the proposed framework is independent of specific model architectures and can be integrated with a wide range of representation learning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。