PCLVis帮助用户分析大规模模拟中的通信延迟,无需管理员权限。
PCLVis: Visual Analytics of Process Communication Latency in Large-Scale Simulation
- 基于MPI通信数据构建进程相关性树定位延迟事件
- 通过通信依赖图和滑动窗口抽象识别延迟传播路径
- 用通信状态图标可视化进程状态,支持交互式诊断
大规模超算模拟已成为重要工具,但并行进程间的通信开销严重制约其可扩展性。现有分析方法依赖物理链路层信息,仅管理员可访问。本文提出PCLVis框架,使普通用户也能分析进程通信延迟(PCL)事件。该框架不依赖物理层数据,而是利用MPI通信数据:首先构建进程相关性树,将高相关性进程聚类;其次建立基于通信依赖的有向无环图(DAG),结合滑动窗口算法生成延迟事件抽象,支持用户交互探索延迟事件的时间演化;同时设计通信状态图标(CS-Glyph)可视化每个进程的收发消息与负载均衡状态,叶节点可展开查看详情;最后提出一种延迟事件归因策略,助力用户优化模拟。在TH-1A超算上的多个模拟任务中验证了该框架的有效性,显著提升用户调优效率。
原文摘要 · Abstract (English)
Large-scale simulations on supercomputers have become important tools for users. However, their scalability remains a problem due to the huge communication cost among parallel processes. Most of the existing communication latency analysis methods rely on the physical link layer information, which is only available to administrators. In this paper, a framework called PCLVis is proposed to help general users analyze process communication latency (PCL) events. Instead of the physical link layer information, the PCLVis uses the MPI process communication data for the analysis. First, a spatial PCL event locating method is developed. All processes with high correlation are classified into a single cluster by constructing a process-correlation tree. Second, the propagation path of PCL events is analyzed by constructing a communication-dependency-based directed acyclic graph (DAG), which can help users interactively explore a PCL event from the temporal evolution of a located PCL event cluster. In this graph, a sliding window algorithm is designed to generate the PCL events abstraction. Meanwhile, a new glyph called the communication state glyph (CS-Glyph) is designed for each process to show its communication states, including its in/out messages and load balance. Each leaf node can be further unfolded to view additional information. Third, a PCL event attribution strategy is formulated to help users optimize their simulations. The effectiveness of the PCLVis framework is demonstrated by analyzing the PCL events of several simulations running on the TH-1A supercomputer. By using the proposed framework, users can greatly improve the efficiency of their simulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。