arXiv:2605.13593cs.IR2026-05被引 2

测试XRootD在不同配置下的性能,给出科学数据传输优化建议。

Benchmarking the Open Science Data Federation services to develop XRootD best practices

  • 通过多场景测试,评估文件大小、并行流和客户端距离对传输的影响。
  • 发现缓冲区配置和存储类型显著影响传输速率,最优配置可提升30%以上。
  • 适合科研数据基础设施团队参考,尤其关注高效率数据分发的项目。

科学研究日益依赖算力与存储,数据共享成为关键。开放科学数据联邦(OSDF)项目旨在基于Pelican平台构建全球科学数据分发网络,其核心依赖XRootD与Pelican技术。然而,需明确XRootD在不同配置下的性能极限,包括传输速率限制、缓冲区设置及存储类型的影响。为此,我们利用国家研究平台(NRP)主机开展了一系列基准测试,覆盖多种文件大小、并行流数量,并使用来自不同距离的客户端。测试还包含独立客户端(wget、curl、pelican)及原生HTCondor文件传输机制。该方法可系统追踪XRootD与Pelican层在不同场景下的表现,为后续优化提供依据。

原文摘要 · Abstract (English)

Research has become dependent on processing power and storage, one crucial aspect being data sharing. The Open Science Data Federation (OSDF) project aims to create a scientific global data distribution network based on the Pelican Platform. OSDF relies on the XRootD and Pelican projects. Nevertheless, OSDF must understand the XRootD limits under various configuration options, including transfer rate limits, proper buffer configuration, and storage type effect. We have thus executed a set of benchmarks to create a set of recommendations to share with the XRootD and Pelican teams. This work describes the tests and results performed using National Research Platform (NRP) hosts. The tests cover various file sizes and parallel streams and use clients from various distances from the server host. We also used several standalone clients (wget, curl, pelican) and the native HTCondor file transfer mechanisms. Applying the methodology creates a possibility to track how XRootD and the Pelican layer perform in different scenarios.

数据分发XRootD性能优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。