通过不确定性建模提升文本嵌入的集成效果
Uncertainty-driven Embedding Convolution
- 将确定性嵌入转为概率嵌入，基于不确定性自适应加权
- 在多个基准上提升性能与鲁棒性，优于传统集成方法
- 适合需要可靠嵌入表示的下游任务，如问答与分类
文本嵌入是现代自然语言处理流程的核心组件。尽管已有众多嵌入模型，但没有单一模型能在所有领域和任务中持续领先。这种差异性促使使用集成技术来融合互补优势。然而，现有大多数集成方法基于确定性嵌入，未能考虑模型特异性不确定性，限制了其在下游应用中的鲁棒性和可靠性。为此，我们提出不确定性驱动的嵌入卷积（UEC）。UEC首先以事后方式将确定性嵌入转换为概率嵌入；然后基于不确定性计算自适应集成系数，该系数来自一个理论合理的代理损失公式。此外，UEC采用一种考虑不确定性的相似度函数，直接将不确定性融入相似度评分，提供一种理论严谨且高效的分布距离替代方案。在多个不同基准上的大量实验表明，通过合理建模不确定性，UEC能持续提升性能与鲁棒性。
原文摘要 · Abstract (English)
Text embeddings are essential components in modern NLP pipelines. Although numerous embedding models have been proposed, no single model consistently dominates across domains and tasks. This variability motivates the use of ensemble techniques to combine complementary strengths. However, most existing ensemble methods operate on deterministic embeddings and fail to account for model-specific uncertainty, limiting their robustness and reliability in downstream applications. To address these limitations, we propose Uncertainty-driven Embedding Convolution (UEC). UEC first transforms deterministic embeddings into probabilistic ones in a post-hoc manner. It then computes adaptive ensemble coefficients based on embedding uncertainty, derived from a principled surrogate-loss formulation. Additionally, UEC employs an uncertainty-aware similarity function that directly incorporates uncertainty into the similarity scoring, providing a theoretically grounded and efficient surrogate to distributional distances. Extensive experiments on diverse benchmarks demonstrate that UEC consistently improves both performance and robustness by leveraging principled uncertainty modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。