用问答蓝图提升多语言表格生成的可追溯性,但翻译误差限制了效果。
The Challenge of Achieving Attributability in Multilingual Table-to-Text Generation with Question-Answer Blueprints
- 用问答蓝图作为中间规划,引导模型生成更忠实于表格内容的文本。
- 英文单语训练下蓝图显著提升可追溯性,多语言场景中未见增益。
- 机器翻译蓝图引入误差,模型也未能有效依赖自动生成的蓝图。
多语言自然语言生成(NLG)因低资源语言训练数据匮乏而面临挑战。尽管某些低资源语言全球使用者可达数千万,仍需提升其NLG工具性能。表格转文本任务是衡量模型推理能力的良好指标,但在多语言环境下极难实现输出可追溯性。已有研究表明,问答(QA)蓝图等中间规划方法可提升摘要任务的可追溯性。本文探究该方法在多语言表格转文本任务中的有效性,扩展了包含非洲语言的挑战性数据集TaTA,加入QA蓝图。对序列到序列模型进行微调,对比有无蓝图的情况。结果显示:仅在英语数据上微调与评估时,蓝图能提升性能;但在多语言设置中未见收益。原因在于将蓝图从英语机器翻译至目标语言时产生误差,且模型未能充分依赖所生成的蓝图。本文深入分析了其中挑战。
原文摘要 · Abstract (English)
Multilingual Natural Language Generation (NLG) is challenging due to the lack of training data for low-resource languages. However, some low-resource languages have up to tens of millions of speakers globally, making it important to improve NLG tools for them. Table-to-Text NLG is an excellent measure of models' reasoning abilities but is very challenging in the multilingual setting. System outputs are often not attributable, or faithful, to the data in the source table. Intermediate planning techniques like Question-Answer (QA) blueprints have been shown to improve attributability on summarisation tasks. This work explores whether QA blueprints make multilingual Table-to-Text outputs more attributable to the input tables. This paper extends the challenging multilingual Table-to-Text dataset, TaTA, which includes African languages, with QA blueprints. Sequence-to-sequence language models are then finetuned on this dataset, with and without blueprints. Results show that QA blueprints improve performance for models finetuned and evaluated only on English examples, but do not demonstrate gains in the multilingual setting. This is due to inaccuracies in machine translating the blueprints from English into target languages when generating the training data, and models failing to rely closely on the blueprints they generate. An in-depth analysis is conducted on why this is challenging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。