构建首个金融领域文档级机器翻译测试集，突破传统句对齐限制。
DOLFIN -- Document-Level Financial test set for Machine Translation
- 以段落为单位构建非严格对齐的金融文档数据集
- 每语言对平均含1950个对齐段落，覆盖5个语言对
- 适合评估模型在金融文本中的上下文理解能力
尽管文档级机器翻译（MT）研究日益活跃，但专门针对该任务的测试集仍十分稀缺。现有测试集多集中于通用领域，难以覆盖法律、金融等专业领域；且虽具文档级特征，仍沿用句子级对齐逻辑，无法体现信息重组等复杂现象。为此，本文提出新型测试集DOLFIN，基于专业金融文档构建，通过摒弃严格句对齐，以段落为单位呈现数据，推动真正意义上的文档级翻译评估。测试集包含5个语言对，平均每对有1950个对齐段落。我们提供详细的数据采集流程，可供其他文档级数据集对齐参考。通过评估多个模型，结果表明该测试集能有效区分上下文敏感与无关模型，并揭示模型在金融文本翻译中的不足。该数据集已公开供社区使用。
原文摘要 · Abstract (English)
Despite the strong research interest in document-level Machine Translation (MT), the test sets dedicated to this task are still scarce. The existing test sets mainly cover topics from the general domain and fall short on specialised domains, such as legal and financial. Also, in spite of their document-level aspect, they still follow a sentence-level logic that does not allow for including certain linguistic phenomena such as information reorganisation. In this work, we aim to fill this gap by proposing a novel test set: DOLFIN. The dataset is built from specialised financial documents, and it makes a step towards true document-level MT by abandoning the paradigm of perfectly aligned sentences, presenting data in units of sections rather than sentences. The test set consists of an average of 1950 aligned sections for five language pairs. We present a detailed data collection pipeline that can serve as inspiration for aligning new document-level datasets. We demonstrate the usefulness and quality of this test set by evaluating a number of models. Our results show that the test set is able to discriminate between context-sensitive and context-agnostic models and shows the weaknesses when models fail to accurately translate financial texts. The test set is made public for the community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。