针对量子编程代码生成难题,构建专用知识库提升大模型准确性。
PennySynth: RAG-Driven Data Synthesis for Automated Quantum Code Generation

- 用1.3万条真实量子代码构建知识库,结合检索增强生成。
- 在三年竞赛题上通过率超主流模型28个百分点,结构错误大幅减少。
- 专设量子代码相似度评估指标,适合量子算法开发者使用。
量子编程框架日益复杂,现有大语言模型在处理专用任务时常出现虚构门名称、配置错误及电路结构失效等问题。本文提出PennySynth,一种基于检索增强生成的框架,其核心是构建包含13,389条PennyLane指令-代码对的精选知识库,数据来源涵盖官方仓库、社区GitHub与QHack竞赛档案,并经三阶段提取、验证与去重流程处理。PennySynth采用基于st-codesearch-distilroberta-base的代码感知嵌入策略,使平均检索余弦相似度从0.45提升至0.726。在覆盖2022至2024年共74个挑战的QHack测试集上,PennySynth pass@5分别为64%、68%和52%,相较无检索的Claude Sonnet 4.6分别提升28、25、28个百分点。此外,引入量子适配的CodeBLEU指标,重点加权qml.*等代码模式,证明结构相似性与功能正确性反映量子代码质量的不同维度。受控消融实验表明,代码感知嵌入是检索性能的关键驱动因素,而数据集扩展与多源融合在高精度检索下进一步带来增益。
原文摘要 · Abstract (English)
The growing complexity of quantum programming frameworks has exposed a critical limitation in existing large language model (LLM)-based code assistants: general-purpose models hallucinate PennyLane-specific gate names, misplace device configurations, and produce structurally invalid circuits when faced with specialized quantum coding challenges. We present PennySynth, a retrieval-augmented generation framework that addresses this gap by conditioning LLM inference on a curated knowledge base of 13,389 PennyLane instruction-code pairs, built via a three-stage extraction, verification, and deduplication pipeline over official PennyLane repositories, community GitHub sources, and QHack competition archives. PennySynth introduces a code-aware embedding strategy using st-codesearch-distilroberta-base, trained for natural-language-to-code retrieval, increasing average retrieval cosine similarity from 0.45 to 0.726 compared to a general-purpose baseline. Evaluated across 74 challenges spanning three years of the QHack competition (2022, 2023, 2024), PennySynth achieves 64%, 68%, and 52% pass@5 on QHack 2022, 2023, and 2024, respectively, improving over Claude Sonnet 4.6 without retrieval by +28, +25, and +28 percentage points. We further introduce a quantum-adapted CodeBLEU metric that upweights qml.* token patterns and show that structural code similarity and functional correctness capture distinct aspects of quantum code quality. Controlled ablations reveal that code-aware embeddings are the primary driver of retrieval performance, while dataset expansion and source composition provide additional gains when retrieval quality is sufficiently precise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。