用源语言数据提升低资源目标语言生成的语义准确性。
Source-Grounded Semantic Reinforcement Learning for Low-Resource Target-Language Generation

- 用跨语言语义奖励模型,让源语言单语数据指导目标语言生成。
- 在中文转泰语任务中,语义准确率和事实覆盖率显著优于零样本微调。
- 适合低资源语言生成场景,尤其适用于缺乏平行语料的翻译任务。
低资源目标语言生成常受限于平行数据稀缺,而高资源源语言的单语数据虽丰富却难以通过标准监督微调利用。我们提出源语言基础语义强化学习(SG-SRL),将源语言单语数据转化为跨语言语义监督,用于目标语言生成。SG-SRL在源语言数据上进行无参考强化学习,使用基于跨语言重排序器的语义奖励模型,评估源输入与目标语言生成之间的语义相关性。尽管该方法易导致冗长化奖励欺骗,但通过小规模平行语料的轻量恢复阶段,可有效恢复流畅性、简洁性和任务格式,同时保留语义增益。在中-泰生成任务上的实验表明,相比冷启动监督微调,SG-SRL显著提升了语义根基与事实覆盖。对长文本迁移及藏语嵌入奖励的分析进一步揭示了其泛化能力,证明在真实低资源环境下,编码器型语义奖励可替代大模型重排序器。
原文摘要 · Abstract (English)
Low-resource target-language generation is often limited by scarce parallel data, while high-resource source-language monolingual data is abundant but difficult to use with standard supervised fine-tuning. We propose Source-Grounded Semantic Reinforcement Learning (SG-SRL), a resource-utilization framework that converts source-language monolingual data into cross-lingual semantic supervision for target-language generation. SG-SRL performs reference-free reinforcement learning (RL) on source-language data using a cross-lingual semantic reward model, instantiated by a cross-lingual reranker that scores the semantic relevance between the source input and the target-language generation. While this induces severe verbosity-based reward hacking, a lightweight recovery stage using a small parallel corpus restores fluency, conciseness, and task format while preserving the semantic gains. Experiments on Chinese-to-Thai generation show that SG-SRL improves semantic grounding and factual coverage over cold-start SFT. Additional analyses on long-form transfer and Tibetan embedding-based rewards clarify the generalization behavior of SG-SRL and show that an encoder-based semantic reward can substitute for an LLM-based reranker in a realistic low-resource language setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。