用强化学习让小模型零样本生成学术知识图谱查询,效果接近有标注训练。
Text-to-SPARQL Generation with Reinforcement Learning: A GRPO-based Approach on DBLP
- 用基于结果的奖励机制训练小模型,无需标注查询序列。
- 在DBLP数据集上执行准确率达78.6%,超越零样本基线。
- 适合无标注数据场景,尤其适用于学术问答系统构建。
知识图谱问答旨在将自然语言问题转化为可执行查询,但现有方法多依赖大模型或完整的黄金查询标注。本研究探讨在无监督条件下,是否可通过基于结果的强化学习训练小型指令微调语言模型实现零样本文本到SPARQL的生成。针对Qwen3-1.7B模型,在DBLP-QuAD数据集上采用组相对策略优化(GRPO),使用结合自然语言问题与实体/关系符号提示的提示模板。训练依赖执行反馈、结构约束和答案级奖励,另设一个基于黄金查询的引导变体。模型在答案级准确率、执行准确率、类别得分及未见模板泛化能力上,与未经修改的零样本基线和监督微调的DoRA基线对比。GRPO显著优于零样本基线,表现出良好泛化能力;而监督的DoRA微调在同一模型规模下取得更高整体准确率。消融分析表明,执行反馈奖励贡献主要提升,额外引导作用有限,说明在缺乏黄金标注时,基于结果的强化学习是可行的训练策略。
原文摘要 · Abstract (English)
Knowledge graph question answering seeks to translate natural language questions into executable queries over knowledge graphs, but existing approaches often rely on large models or full supervision in the form of gold query annotations. This study examines whether reinforcement learning with outcome-based rewards can train a small instruction-tuned language model to perform zero-shot Text-to-SPARQL generation in the scholarly domain. Group-Relative Policy Optimization (GRPO) is applied to the Qwen3-1.7B model on DBLP-QuAD, using prompts that combine natural language questions with symbolic hints about entities and relations. Training relies on execution feedback, structural constraints, and answer-level rewards, with an additional variant that incorporates gold-query-based shaping. The resulting models are compared to the unmodified zero-shot baseline and to a supervised DoRA-finetuned baseline across answer-level accuracy, execution accuracy, category-wise scores, and generalization to held-out templates. GRPO substantially improves over the zero-shot baseline and exhibits competitive generalization, while supervised DoRA finetuning achieves higher overall accuracy on the same model scale. Ablation analyses indicate that execution-based rewards account for most gains, with additional shaping yielding limited additional benefit, suggesting that outcome-based reinforcement learning is a viable training strategy when gold queries are unavailable for token-level supervision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。