评测大模型用不同提示生成并行代码的性能与效率。
From Prompts to Performance: Evaluating LLMs for Task-based Parallel Code Generation
- 用自然语言、串行代码和伪并行代码三种提示生成并行程序。
- 在OpenMP、C++标准并行和HPX框架下测试正确性与可扩展性。
- 发现模型表现受问题复杂度和编程框架影响显著。
大型语言模型（LLM）在代码生成方面表现出色，但在生成高效并行程序方面的能力尚未充分研究。本文探讨了LLM如何从三类输入提示——自然语言问题描述、串行参考实现和并行伪代码——生成基于任务的并行代码。研究聚焦于三个编程框架：OpenMP Tasking、C++标准并行化以及异步多任务运行时HPX，它们提供了不同程度的任务执行抽象与控制。我们评估了生成代码的正确性与可扩展性。结果揭示了LLM在问题复杂度和框架差异下的优势与局限。最后，讨论了这些发现对高性能计算和科学计算中未来LLM辅助开发的意义。
原文摘要 · Abstract (English)
Large Language Models (LLM) show strong abilities in code generation, but their skill in creating efficient parallel programs is less studied. This paper explores how LLMs generate task-based parallel code from three kinds of input prompts: natural language problem descriptions, sequential reference implementations, and parallel pseudo code. We focus on three programming frameworks: OpenMP Tasking, C++ standard parallelism, and the asynchronous many-task runtime HPX. Each framework offers different levels of abstraction and control for task execution. We evaluate LLM-generated solutions for correctness and scalability. Our results reveal both strengths and weaknesses of LLMs with regard to problem complexity and framework. Finally, we discuss what these findings mean for future LLM-assisted development in high-performance and scientific computing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。