用对比奖励模型提升硬件生成质量，训练快8.5倍且更省资源。
HLS-Seek: QoR-Aware Code Generation for High-Level Synthesis via Proxy Comparative Reward Reinforcement Learning
- 通过对比奖励替代真实合成，实现高效低耗的代码优化。
- 在30个核函数中16个延迟最低，9个全面超越现有基线。
- 适合需要快速生成高质量硬件代码的研究者和工程师。
高层次综合（HLS）将算法级C/C++代码编译为硬件，其质量（QoR）由pragma配置和代码结构决定。现有基于大语言模型的HLS方法仅关注功能正确性，忽略QoR。我们发现HLS的强化学习无需绝对合成结果，只需候选方案间的相对比较。据此提出HLS-Seek：一种面向QoR的自然语言到HLS框架，以代理对比奖励模型替代昂贵的合成反馈，达到99.53%的Pareto占优准确率。为防止奖励欺骗，引入不确定性感知的蒙特卡洛丢弃切换机制，仅对低置信度候选调用真实Vitis HLS合成并在线更新代理模型，形成自进化奖励系统。HLS-Seek在HLS-eval上实现81.5%的pass@1语法正确率与81.4%的Func@5，仅用7B参数即超越GPT-5.1等前沿模型，训练速度比真实奖励强化学习快8.5倍。在QoR评估中，于30个核函数中16个达到最低延迟，9个实现帕累托占优。
原文摘要 · Abstract (English)
High-Level Synthesis (HLS) compiles algorithmic C/C++ descriptions into hardware, with Quality of Results (QoR) -- latency and resource utilization -- critically governed by pragma configurations and code structure. Existing LLM-based HLS approaches train for functional correctness but ignore QoR entirely. We observe that reinforcement learning (RL) for HLS does not require absolute synthesis results -- only relative comparisons between candidates. Based on this insight, we propose \textbf{HLS-Seek}, a QoR-aware NL-to-HLS framework that replaces expensive synthesis-in-the-loop RL with a comparative proxy reward model achieving 99.53\% Pareto-dominance accuracy. To prevent reward hacking, we introduce \textit{uncertainty-aware Monte Carlo (MC) dropout switching} that selectively invokes real Vitis HLS synthesis for low-confidence candidates and online updates the proxy, creating a self-improving reward system. HLS-Seek achieves 81.5\% syntax correctness pass@1 and 81.4\% Func@5 on HLS-eval with only 7B parameters, surpassing GPT-5.1 and other frontier models while achieving 8.5$\times$ faster training than real-reward RL. On QoR evaluation, HLS-Seek achieves the lowest latency on 16/30 kernels and Pareto-dominates HLS-specific baselines on 9 kernels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。