用对比奖励提升多语言文本转SQL的语义对齐,小模型也能超越大模型。
Bridging the Semantic Gap: Contrastive Rewards for Multilingual Text-to-SQL with GRPO
- 用分组相对策略优化结合语义对比奖励,让模型更懂用户意图。
- 在七种语言上,语义准确率最高提升至59.14%,执行准确率达87.4%。
- 30亿参数小模型经训练后,性能接近80亿参数大模型,仅需3000条数据。
现有Text-to-SQL方法仅关注可执行查询,忽视了语义对齐问题——包括查询语义与执行结果正确性的双重挑战。跨语言迁移时,执行准确率平均下降6个百分点。本文提出一种新框架,将分组相对策略优化(GRPO)与多语言对比奖励信号结合,在跨语言场景中同时提升任务效率与语义准确性。通过基于语义相似性的奖励机制,增强模型生成的SQL与用户意图的一致性。在七语言MultiSpider数据集上,微调LLaMA-3-3B模型使用GRPO后,执行准确率提升至87.4%(较零样本提升26个百分点),语义准确率提升至52.29%(提升32.86个百分点)。引入对比奖励信号后,平均语义准确率进一步提升至59.14%(+6.85个百分点,越南语最高+10个百分点)。实验表明,仅用3,000条强化学习样本微调的30亿参数模型,执行准确率(88.86%)比零样本80亿模型(81.43%)高出7.43个百分点,语义准确率也接近其水平(59.14% vs. 68.57%),证明对比奖励能有效实现语义对齐,无需大规模训练数据。
原文摘要 · Abstract (English)
Current Text-to-SQL methods are evaluated and only focused on executable queries, overlooking the semantic alignment challenge -- both in terms of the semantic meaning of the query and the correctness of the execution results. Even execution accuracy itself shows significant drops when moving from English to other languages, with an average decline of 6 percentage points across non-English languages. We address these challenges by presenting a new framework that combines Group Relative Policy Optimization (GRPO) within a multilingual contrastive reward signal to enhance both task efficiency and semantic accuracy in Text-to-SQL systems in cross-lingual scenarios. Our method teaches models to obtain better correspondence between SQL generation and user intent by combining a reward signal based on semantic similarity. On the seven-language MultiSpider dataset, fine-tuning the LLaMA-3-3B model with GRPO improved the execution accuracy up to 87.4 percent (+26 pp over zero-shot) and semantic accuracy up to 52.29 percent (+32.86 pp). Adding our contrastive reward signal in the GRPO framework further improved the average semantic accuracy to 59.14 percent (+6.85 pp, up to +10 pp for Vietnamese). Our experiments showcase that a smaller, parameter-efficient 3B LLaMA model fine-tuned with our contrastive reward signal outperforms a much larger zero-shot 8B LLaMA model, with an uplift of 7.43 pp in execution accuracy (from 81.43 percent on the 8B model to 88.86 percent on the 3B model), and nearly matches its semantic accuracy (59.14 percent vs. 68.57 percent) -- all using just 3,000 reinforcement learning training examples. These results demonstrate how we can improve the performance of Text-to-SQL systems with contrastive rewards for directed semantic alignment, without requiring large-scale training datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。