通过清洗数据提升嵌入微调效果，显著改善特定领域检索性能。
Embedding Retrofitting: Data Engineering for better RAG
- 用知识图谱约束调整预训练词向量，提升领域检索精度。
- 去除标签噪声后，检索准确率最高提升6.2%，定量问题提升33.8%。
- 数据预处理质量比算法选择更重要，是决定成败的关键因素。
嵌入微调通过知识图谱约束调整预训练词向量，以提升领域特定检索效果。然而其效果高度依赖知识图谱质量，而图谱质量又受文本预处理影响。本文提出一种数据工程框架，解决真实语料中注释伪影导致的数据质量下降问题。分析显示，话题标签会过度增加知识图谱密度，产生虚假边，破坏微调目标。在含噪图谱上，所有微调方法均出现显著退化（-3.5%至-5.2%，p<0.05）。经预处理后，EWMA微调实现+6.2%提升（p=0.0348），优势集中于定量合成问题（平均+33.8%）。干净与噪声预处理间的差距（10%以上波动）超过不同算法间的差异（3%），确立预处理质量为微调成功的核心决定因素。
原文摘要 · Abstract (English)
Embedding retrofitting adjusts pre-trained word vectors using knowledge graph constraints to improve domain-specific retrieval. However, the effectiveness of retrofitting depends critically on knowledge graph quality, which in turn depends on text preprocessing. This paper presents a data engineering framework that addresses data quality degradation from annotation artifacts in real-world corpora. The analysis shows that hashtag annotations inflate knowledge graph density, leading to creating spurious edges that corrupt the retrofitting objective. On noisy graphs, all retrofitting techniques produce statistically significant degradation ($-3.5\%$ to $-5.2\%$, $p<0.05$). After preprocessing, \acrshort{ewma} retrofitting achieves $+6.2\%$ improvement ($p=0.0348$) with benefits concentrated in quantitative synthesis questions ($+33.8\%$ average). The gap between clean and noisy preprocessing (10\%+ swing) exceeds the gap between algorithms (3\%), establishing preprocessing quality as the primary determinant of retrofitting success.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。