分布式训练可能让大模型训练绕过算力监管，需新手段防范。
Does Distributed Training Undermine Compute Governance?
- 用分散硬件替代数据中心进行大模型训练
- 现有监管难发现隐蔽的分布式训练集群
- 适合关注算力治理与合规的技术政策研究者
算力治理方案通常假设前沿人工智能训练需依赖大型、可检测的计算集群。然而，分布式训练算法的进展使开发者可在非集中式硬件上完成前沿规模训练，无需使用大型数据中心设施。偏好规避监管的开发者可能通过特定硬件组合方式，避开算力登记与监控要求。因此，监管措施必须具备识别和阻止非法分布式训练的能力。本文评估了此类规避行为的可行性，并提出包括举报机制、芯片追踪、审计取证及集群内存与算力阈值等应对策略。
原文摘要 · Abstract (English)
Compute governance proposals often rely on the assumption that frontier AI training requires large, detectable computing clusters. However, recent advances in distributed training algorithms could allow developers to conduct frontier-scale training on distributed agglomerations of hardware, rather than needing large datacenter facilities. Developers who prefer not to be constrained by regulations may structure their hardware in a manner that evades the registration and monitoring requirements associated with compute governance. Therefore, regulations must be designed to detect and prevent illicit distributed training operations. This paper evaluates the feasibility of such evasion and outlines recommended countermeasures, including whistleblowing, chip tracking, forensic accounting, and memory and compute thresholds for clusters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。