不用微调,用检索+智能反馈也能让大模型写出更好量子代码。
Revisiting Quantum Code Generation: Where Should Domain Knowledge Live?
- 用检索增强和智能执行反馈替代领域微调
- 最强模型达85%正确率,比专用微调高35%
- 适合关注可维护性的量子软件开发者
大语言模型在编程自动化中表现突出,尤其在快速演进的量子软件领域。本文研究Qiskit代码生成的专化策略,对比了先前参数微调模型与最新通用大模型结合检索增强生成(RAG)和带执行反馈的智能体推理方法。结果表明,现代通用模型在零样本和检索增强下均达到60-65%的通过率(pass@1),最强模型结合迭代执行反馈可达85%,相比零样本通用模型提升超20%,比参数微调模型高35%以上。智能体反馈带来最稳定提升,尽管增加运行时间;检索增强仅提供适度且依赖模型的增益。说明无需领域微调即可实现高性能,通过推理时增强可实现更灵活、易维护的量子软件辅助开发。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have enabled the automation of an increasing number of programming tasks, including code generation for scientific and engineering domains. In rapidly evolving software ecosystems such as quantum software development, where frameworks expose complex abstractions, a central question is how best to incorporate domain knowledge into LLM-based assistants while preserving maintainability as libraries evolve. In this work, we study specialization strategies for Qiskit code generation using the Qiskit-HumanEval benchmark. We compare a parameter-specialized fine-tuned baseline introduced in prior work against a range of recent general-purpose LLMs enhanced with retrieval-augmented generation (RAG) and agent-based inference with execution feedback. Our results show that modern general-purpose LLMs consistently outperform the parameter-specialized baseline. While the fine-tuned model achieves approximately 47% pass@1 on Qiskit-HumanEval, recent general-purpose models reach 60-65% under zero-shot and retrieval-augmented settings, and up to 85% for the strongest evaluated model when combined with iterative execution-feedback agents -representing an improvement of more than 20% over zero-shot general-purpose performance and more than 35% over the parameter-specialized baseline. Agentic execution feedback yields the most consistent improvements, albeit at increased runtime cost, while RAG provides modest and model-dependent gains. These findings indicate that performance gains can be achieved without domain-specific fine-tuning, instead relying on inference-time augmentation, thereby enabling a more flexible and maintainable approach to LLM-assisted quantum software development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。