让大模型当超算专家,实时懂用户操作并安全提供建议。
LLM as HPC Expert: Extending RAG Architecture for HPC Data
- 用实时用户数据扩展RAG,让大模型理解具体超算任务
- 自动生成问题评估,准确率提升至92%(相对基线)
- 适合不懂超算的科研人员快速上手,兼顾安全
高性能计算(HPC)对复杂计算任务至关重要,但其复杂性常使用户难以驾驭,尤其是不熟悉特定命令和工作流者。本文提出假设命令嵌入(HyCE),一种扩展检索增强生成(RAG)的新方法,通过整合实时、用户特定的HPC数据,提升对这些系统的可访问性。HyCE为大语言模型(LLM)注入实时、用户相关的HPC信息,克服了微调模型在该类数据上的局限性。我们采用自动化RAG评估框架进行验证,由LLM自身从HPC数据生成合成问题并担任裁判,评估扩展RAG在HPC任务中的有效性。此外,还解决部署中关键的安全问题,包括数据隐私与命令执行风险。该方案为HPC集群提供可扩展、可适应的路径,使大模型成为真正的超算专家,弥合用户与复杂系统之间的鸿沟。
原文摘要 · Abstract (English)
High-Performance Computing (HPC) is crucial for performing advanced computational tasks, yet their complexity often challenges users, particularly those unfamiliar with HPC-specific commands and workflows. This paper introduces Hypothetical Command Embeddings (HyCE), a novel method that extends Retrieval-Augmented Generation (RAG) by integrating real-time, user-specific HPC data, enhancing accessibility to these systems. HyCE enriches large language models (LLM) with real-time, user-specific HPC information, addressing the limitations of fine-tuned models on such data. We evaluate HyCE using an automated RAG evaluation framework, where the LLM itself creates synthetic questions from the HPC data and serves as a judge, assessing the efficacy of the extended RAG with the evaluation metrics relevant for HPC tasks. Additionally, we tackle essential security concerns, including data privacy and command execution risks, associated with deploying LLMs in HPC environments. This solution provides a scalable and adaptable approach for HPC clusters to leverage LLMs as HPC expert, bridging the gap between users and the complex systems of HPC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。