用大模型驱动的混合遗传算法，提升少样本语法规则生成效果
HyGenar: An LLM-Driven Hybrid Genetic Algorithm for Few-Shot Grammar Generation
- 结合大模型与遗传算法，动态优化语法规则生成
- 在540个挑战任务上显著提升语法和语义正确性
- 适合需要精准结构化输出的文本/代码生成场景
语法在自然语言处理和文本/代码生成中至关重要，用于定义语法结构、构建解析器并引导结构化输出。尽管大语言模型（LLMs）在多个领域表现优异，但其推断和生成语法的能力尚未充分探索。本文研究并提升LLMs在少样本语法规则生成中的能力，即从少量正负例中推导并生成以巴科斯-诺尔范式（Backus-Naur Form）表示的语法规则。为此，我们构建了一个包含540个结构化语法规则生成挑战的新数据集，设计了6项评估指标，并对8种不同LLMs进行了评测。结果表明，现有LLMs在语法规则生成上表现不佳。为解决此问题，我们提出一种大模型驱动的混合遗传算法HyGenar，可显著提升各类LLMs生成语法规则的语法和语义正确性。
原文摘要 · Abstract (English)
Grammar plays a critical role in natural language processing and text/code generation by enabling the definition of syntax, the creation of parsers, and guiding structured outputs. Although large language models (LLMs) demonstrate impressive capabilities across domains, their ability to infer and generate grammars has not yet been thoroughly explored. In this paper, we aim to study and improve the ability of LLMs for few-shot grammar generation, where grammars are inferred from sets of a small number of positive and negative examples and generated in Backus-Naur Form. To explore this, we introduced a novel dataset comprising 540 structured grammar generation challenges, devised 6 metrics, and evaluated 8 various LLMs against it. Our findings reveal that existing LLMs perform sub-optimally in grammar generation. To address this, we propose an LLM-driven hybrid genetic algorithm, namely HyGenar, to optimize grammar generation. HyGenar achieves substantial improvements in both the syntactic and semantic correctness of generated grammars across LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。