让跨文档词对齐更准，靠结构约束缩小搜索范围
Scaling Unsupervised Word Alignment to Documents via Structural Constraints
- 用粗到细策略，只在语义相近区域找词对应
- 跨语言对齐错误率从0.412降到0.326，提升明显
- 无需分句或句级对齐，适合长文档任务
词对齐传统上针对句子，但越来越多跨语言任务需要文档级对应。尽管多语言嵌入模型能处理长文本，直接将句级算法用于文档会导致性能下降。为此，我们提出CTFAlign，一种轻量、无需训练的文档级词对齐方法。该方法采用粗到细的精炼策略，将对齐搜索空间限制在语义相似区域。此外，我们还引入MDPAlign，通过主对角线先验按位置约束对齐，实现更简单的替代方案。两者均直接作用于完整文档，无需依赖分句或句级对齐。我们在六组不同语言对（涵盖类型差异、资源丰富度和文档长度）上评估这些方法。三种模型平均下，CTFAlign将词对齐错误率从0.412降至0.326。这种提升可迁移至下游任务，在文档级翻译覆盖率和语义差异识别上带来改进。我们已发布CTFAlign为Python包，并公开代码与实验数据以供复现。
原文摘要 · Abstract (English)
Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual embedding models can encode long inputs, we show that applying algorithms designed for sentences directly to documents leads to performance degradation. To address this, we introduce CTFAlign, a lightweight, training-free approach for document-level word alignment. CTFAlign applies a coarse-to-fine refinement strategy that restricts the alignment search space to semantically similar regions. Additionally, we introduce MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior. Both approaches operate directly on full documents without relying on sentence segmentation or sentence alignment. We evaluate these methods across six language pairs varying in typological distance, resourcedness, and document length. Averaged over three models, CTFAlign reduces word alignment error rate from 0.412 to 0.326. These gains transfer downstream, leading to improvements in document-level translation coverage evaluation and recognition of semantic differences. We release CTFAlign as a Python package and make the code and data to reproduce our experiments publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。