用多模态模型和大模型优化硬件综合,更快找到最佳设计方案。
MPM-LLM4DSE: Reaching the Pareto Frontier in HLS with Multimodal Learning and LLM-Driven Exploration
- 融合行为描述与数据流图特征,提升设计质量预测精度
- 大模型结合提示工程生成配置,平均性能提升39.90%
- 适合数字电路设计、自动化综合领域研究者使用
高层次综合(HLS)设计空间探索(DSE)旨在复杂参数配置空间中寻找帕累托最优设计。为加速HLS DSE,通常采用图神经网络(GNN)作为替代模型预测结果质量(QoR)指标,配合多目标优化算法推进探索。然而,现有基于GNN的预测方法难以充分捕捉行为描述中的丰富语义特征,传统多目标优化算法也未显式引入参数指令对QoR影响的领域知识。为此,本文提出MPM-LLM4DSE框架,整合多模态预测模型(MPM),同步融合行为描述与控制/数据流图特征;并引入大语言模型(LLM)作为优化器,辅以针对参数指令影响的定制化提示工程方法(LLM4DSE),引导生成高质量配置。实验表明,所提多模态模型相比当前最优方法ProgSG,预测性能最高提升10.25倍;在DSE任务中,该框架相较已有方法平均性能提升39.90%,验证了提示工程的有效性。代码与模型开源:https://github.com/wslcccc/MPM-LLM4DSE。
原文摘要 · Abstract (English)
High-Level Synthesis (HLS) design space exploration (DSE) seeks Pareto-optimal designs within expansive pragma configuration spaces. To accelerate HLS DSE, graph neural networks (GNNs) are commonly employed as surrogates for HLS tools to predict quality of results (QoR) metrics, while multi-objective optimization algorithms expedite the exploration. However, GNN-based prediction methods may not fully capture the rich semantic features inherent in behavioral descriptions, and conventional multi-objective optimization algorithms often do not explicitly account for the domain-specific knowledge regarding how pragma directives influence QoR. To address these limitations, this paper proposes the MPM-LLM4DSE framework, which incorporates a multimodal prediction model (MPM) that simultaneously fuses features from behavioral descriptions and control and data flow graphs. Furthermore, the framework employs a large language model (LLM) as an optimizer, accompanied by a tailored prompt engineering methodology. This methodology incorporates pragma impact analysis on QoR to guide the LLM in generating high-quality configurations (LLM4DSE). Experimental results demonstrate that our multimodal predictive model significantly outperforms state-of-the-art work ProgSG by up to 10.25$\times$. Furthermore, in DSE tasks, the proposed LLM4DSE achieves an average performance gain of 39.90\% over prior methods, validating the effectiveness of our prompting methodology. Code and models are available at https://github.com/wslcccc/MPM-LLM4DSE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。