通过线性擦除概念，有效去除文档嵌入中的源语言等干扰因素。
The Medium Is Not the Message: Deconfounding Document Embeddings via Linear Concept Erasure
- 提出线性概念擦除方法，从编码器表示中移除已知混淆因子
- 各类任务与嵌入变体下相似度和聚类性能显著提升
- 去偏后模型在分布外任务上表现稳定，未造成性能损失
基于嵌入的文本序列相似性度量不仅受我们关注的内容维度影响，还可能受文本来源、语言等伪属性的干扰。这些文档混淆因子在多个应用中带来问题，尤其在需要跨语料库文本聚合的任务中。本文表明，一种去偏算法通过从编码器表示中移除可观测混淆因子信息，可在极低计算成本下显著降低此类偏差。在所有评估的嵌入变体和任务中，相似度与聚类指标均得到改善，常有大幅提升。有趣的是，分布外基准上的性能未受影响，表明嵌入本身未被破坏。
原文摘要 · Abstract (English)
Embedding-based similarity metrics between text sequences can be influenced not just by the content dimensions we most care about, but can also be biased by spurious attributes like the text's source or language. These document confounders cause problems for many applications, but especially those that need to pool texts from different corpora. This paper shows that a debiasing algorithm that removes information about observed confounders from the encoder representations substantially reduces these biases at a minimal computational cost. Document similarity and clustering metrics improve across every embedding variant and task we evaluate -- often dramatically. Interestingly, performance on out-of-distribution benchmarks is not impacted, indicating that the embeddings are not otherwise degraded.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。