用强化学习提升大模型生成建筑规范代码的准确性
Reinforcement learning to improve large language model-based automated code compliance systems
- 先微调再用分组相对策略优化，提升代码骨架生成精度
- 相比基线，树编辑距离降低23.8%，字符级距离降38.6%
- 零样本下超越Claude、GPT等主流模型，适合建筑合规自动化
基于大语言模型（LLM）的建筑规范自动化合规系统易生成错误或幻觉化的可计算规则。本文提出P4IR框架，分两阶段：首先通过监督微调（SFT）注入领域知识，再采用分组相对策略优化（GRPO）提升高阶代码骨架的生成准确性。相较于SFT基线，该方法在树编辑距离和词元级莱文斯坦距离上分别降低23.8%和38.6%。对比分析显示，在零样本设置下，该方法通过少样本提示优于Claude Opus和Sonnet 4.5、GPT-5.2、Qwen-3-Max及GLM-4.7，在代码结构与语义层面表现更优。此外，GRPO阶段还带来小幅但统计显著的假阳性减少。通过结合SFT与GRPO直接优化领域目标，该方法为构建更准确可靠的LLM驱动合规系统提供可行路径。
原文摘要 · Abstract (English)
Large language model (LLM)-based approaches for automated code compliance (ACC) of building regulations are prone to generating incorrect and hallucinated computer-processable rules. This paper introduces P4IR, a two-stage framework that uses supervised fine-tuning (SFT) to instill domain knowledge in an LLM, followed by Group Relative Policy Optimization (GRPO) to improve the accuracy of the generated intermediate representations in the form of high-level code skeletons. The framework achieved reductions of up to 23.8% and 38.6% in tree edit distance and token-level Levenshtein distance respectively, relative to the SFT baselines. Comparative analysis demonstrates that this approach in a zero-shot setting outperforms leading LLMs in both code structure and semantics, specifically Claude Opus and Sonnet 4.5, GPT-5.2, Qwen-3-Max, and GLM-4.7, evaluated via few-shot prompting. Additionally, the GRPO stage produced a small yet statistically significant reduction in false positives. By combining SFT with GRPO to optimize directly for domain-specific objectives, this approach offers a path toward more accurate and reliable LLM-based ACC systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。