一个模型搞定多语言词与句级对齐,省去重复部署麻烦。
OmniAlign: A Unified Multilingual Aligner for Word and Sentence Alignment
- 用统一轻量模型实现词与句级对齐,基于上下文相似度矩阵生成词对齐。
- 在多个基准上表现接近顶尖水平,跨语言对齐泛化能力强。
- 适合需要处理长文本和多语言平行语料的研究者与工程师。
跨语言序列对齐是构建与利用平行语料库的基础,涵盖从文档、句子到词和子词的映射。现有工具通常只针对单一粒度,导致使用者在多语言和长文本场景下需分别部署词级与句级对齐系统。本文提出 OmniAlign,一个统一的多语言对齐器,仅用一个轻量级模型即可支持词级与句级对齐。其基于编码器架构,具备强长文本建模能力,通过上下文化标记相似度矩阵推导词对齐,并结合句向量与动态规划实现文档级 $m$--$n$ 句子对齐。为平衡细粒度对齐精度与句表示质量,采用四阶段训练流程:对齐导向的持续预训练、自监督学习、人工标注数据上的有监督微调,以及从强大多语言教师模型中蒸馏句向量。实验表明,OmniAlign 在词与句对齐基准上均达到高度竞争性性能,且对未见语言对具有良好泛化能力。令人惊讶的是,后期在短文本上进行有监督微调可进一步提升对齐质量,同时保留前期训练获得的长文本理解能力,使模型在长文本词对齐任务中依然稳健。
原文摘要 · Abstract (English)
Cross-lingual sequence alignment is fundamental for building and exploiting parallel corpora, spanning mappings from documents and sentences down to words and subwords. Existing tools, however, typically specialize in a single granularity, so practitioners often need separate systems for word- and sentence-level alignment---especially in multilingual and long-text settings. We present OmniAlign, a unified multilingual aligner that supports both word-level and sentence-level alignment with a single lightweight model. Built on an encoder-only backbone with strong long-context modeling, OmniAlign induces word alignments from contextualized token similarity matrices, and obtains document-level $m$--$n$ sentence alignments via sentence embeddings combined with dynamic programming. To balance fine-grained alignment accuracy and sentence-representation quality, we use a four-stage training pipeline: alignment-oriented continued pre-training, self-supervised learning, supervised fine-tuning on human annotations, and sentence-embedding distillation from a strong multilingual teacher. Experiments show that OmniAlign achieves highly competitive performance on both word- and sentence-alignment benchmarks and generalizes well to unseen language pairs. Surprisingly, later-stage supervised fine-tuning on short texts further improves alignment quality while retaining the long-context understanding acquired in earlier training, keeping the model robust on long-text word alignment. \normalsize {\color{blue}\textbf{Code}: https://github.com/MilkDargon/OmniAlign}\par {\color{blue}\textbf{Model}: https://huggingface.co/WPS-Qingqiu/OmniAlign}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。