用自动生成数据提升结构化数据推理能力，无需人工标注。
TARGA: Targeted Synthetic Data Generation for Practical Reasoning over Structured Data
- 基于问题实体关系动态生成高相关性合成数据
- 7B模型在GrailQA上提升7.7分，KBQA-Agent上提升12.2分
- 适合需要少样本、强泛化的知识库问答场景
语义解析将自然语言问题转化为逻辑形式，在结构化环境推理中至关重要。然而现有方法面临两大挑战：依赖大量人工标注数据，且对未见样本泛化能力差。为此，我们提出目标导向的合成数据生成框架TARGA，无需人工标注即可动态生成高相关性合成数据。从问题中的关键实体和关系出发，通过分层扩展与跨层组合探测潜在相关查询，并为这些查询生成对应自然语言问题，作为上下文学习的合成示范。在多个知识库问答（KBQA）数据集上的实验表明，仅使用7B参数模型，TARGA显著优于现有非微调方法（使用闭源模型），在GrailQA上F1分数提升7.7，在KBQA-Agent上提升12.2。此外，TARGA在非独立同分布设置下仍表现出更优的样本效率、鲁棒性和泛化能力。
原文摘要 · Abstract (English)
Semantic parsing, which converts natural language questions into logic forms, plays a crucial role in reasoning within structured environments. However, existing methods encounter two significant challenges: reliance on extensive manually annotated datasets and limited generalization capability to unseen examples. To tackle these issues, we propose Targeted Synthetic Data Generation (TARGA), a practical framework that dynamically generates high-relevance synthetic data without manual annotation. Starting from the pertinent entities and relations of a given question, we probe for the potential relevant queries through layer-wise expansion and cross-layer combination. Then we generate corresponding natural language questions for these constructed queries to jointly serve as the synthetic demonstrations for in-context learning. Experiments on multiple knowledge base question answering (KBQA) datasets demonstrate that TARGA, using only a 7B-parameter model, substantially outperforms existing non-fine-tuned methods that utilize close-sourced model, achieving notable improvements in F1 scores on GrailQA(+7.7) and KBQA-Agent(+12.2). Furthermore, TARGA also exhibits superior sample efficiency, robustness, and generalization capabilities under non-I.I.D. settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。