通过拆解程序与语句，提升小模型在低资源语言下的语义解析准确率。
Diverse In-Context Example Selection After Decomposing Programs and Aligned Utterances Improves Semantic Parsing
- 将程序树分解为片段，匹配对应自然语言语句。
- 在多个基准上提升语义解析准确率，小模型效果更显著。
- 适合低资源语言、小模型及复杂程序场景使用。
大语言模型被广泛用于将自然语言语句转换为结构化程序，即语义解析。与原子标签或词元序列不同，程序天然以抽象语法树（AST）形式表示。这种结构化表示带来了上下文示例（ICE）设计与选择的新挑战。本文提出将可用的ICE树分解为片段，部分片段可能更适合解决测试实例。接着，利用带有语法约束提示的额外大模型调用，自动将这些片段映射到对应的自然语言语句。最后，改进并扩展一种近期的多样化上下文示例选择方法，使其适用于完整及碎片化的示例。我们在多个主流语义解析基准上评估了系统SCUD4ICL，结果表明所提出的分解式多样化示例方法带来明显准确率提升。该方法在小型语言模型、包含较大标注树的示例池以及低资源语言中尤其有效。
原文摘要 · Abstract (English)
LLMs are increasingly used as seq2seq translators from natural language utterances to structured programs, a process called semantic interpretation. Unlike atomic labels or token sequences, programs are naturally represented as abstract syntax trees (ASTs). Such structured representation raises novel issues related to the design and selection of in-context examples (ICEs) presented to the LLM. We focus on decomposing the pool of available ICE trees into fragments, some of which may be better suited to solving the test instance. Next, we propose how to use (additional invocations of) an LLM with prompted syntax constraints to automatically map the fragments to corresponding utterances. Finally, we adapt and extend a recent method for diverse ICE selection to work with whole and fragmented ICE instances. We evaluate our system, SCUD4ICL, on popular diverse semantic parsing benchmarks, showing visible accuracy gains from our proposed decomposed diverse demonstration method. Benefits are particularly notable for smaller LLMs, ICE pools having larger labeled trees, and programs in lower resource languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。