用两台桌面级显卡搭建远程分布式大模型训练系统，验证了小实验室的可行性。
Dual-Node NVIDIA DGX Spark over Tailscale: A Remote-Access Testbed for Distributed LLM Training and Cyber-Threat-Intelligence Fine-Tuning
- 两台DGX Spark通过Tailscale远程连接，用200G光纤实现高效分布式训练。
- 4天内处理6.53亿词元，单步耗时69.4秒，全局批大小达13.1万词元。
- 首次在小规模设备上完成网络安全知识微调并验证效果提升。
紧凑型AI系统让本地语言模型实验更易获得，但基于桌面级加速器的多节点训练实证仍有限。本报告展示在两个NVIDIA DGX Spark系统间部署分布式NanoChat预训练的原型系统，每台配备GB10 Grace Blackwell SoC和128 GB统一内存，通过Tailscale网格VPN远程管理，并以200 Gb/s QSFP56直连光纤连接。使用PyTorch torchrun、DDP和NCCL，配置为每节点一个进程，深度20的NanoChat模型，每节点本地批大小32，上下文长度2,048词元，全局批大小达131,072词元/步。训练步骤耗时约69.4秒（约1,890词元/秒），四天内共处理约6.53亿词元。记录了链路配置、容器设置、接口绑定、初始评估错误导致的NCCL超时问题、检查点机制及排错经验，为小型实验室提供可复现参考。同时，基于77份CISA通告构建了网络安全微调数据集（338条训练，37条验证对话），并通过一个包含17个问题的独立评估，对比基线SFT检查点与经CTI增强的检查点（由Ollama托管LLM评判）。结果表明，特定领域类别得分提升，通用知识类别略有下降，整体评分从2.06升至2.29（0-10分制）。同一集群还支持400级人工智能课程（CS 426）及针对CompTIA Security+ POGIL活动的查询引擎，证明小型基础设施兼具科研与教学功能。本研究确立可行性，而非效率对标；对比单节点吞吐率系估算，未在相同条件下实测。运行手册与脚本已公开（见代码可用性）。
原文摘要 · Abstract (English)
Compact AI systems make local language-model experimentation increasingly accessible, yet practical evidence for multi-node training on desktop-class accelerators remains limited. This report presents a proof-of-concept deployment of distributed NanoChat pretraining across two NVIDIA DGX Spark systems, each with a GB10 Grace Blackwell system-on-chip and 128 GB of unified memory, administered remotely over a Tailscale mesh VPN and connected for training by a dedicated 200 Gb/s QSFP56 direct fiber link. PyTorch torchrun, DDP, and NCCL were configured with one process per node, a depth-20 NanoChat model, a local batch size of 32 per node, and a 2,048-token context, giving a global batch of 131,072 tokens per step. The run sustained a step time of about 69.4 s (about 1,890 tokens/s), processing about 653 million tokens over four days. We document link configuration, container setup, interface binding, a step-zero evaluation bug that triggered NCCL timeouts, checkpointing, and troubleshooting lessons, as a reproducibility reference for small labs. We also built a cybersecurity fine-tuning dataset from 77 CISA advisories (338 training, 37 validation conversations) and ran a 17-question held-out evaluation comparing a baseline SFT checkpoint against a CTI-augmented checkpoint with an Ollama-hosted LLM judge. CTI-specific categories improved while general-knowledge categories regressed, for a small overall change from 2.06 to 2.29 on a 0-10 scale. The same cluster supports a 400-level AI course (CS 426) and a query engine for CompTIA Security+ POGIL activities in CBS 255, showing modest local infrastructure can serve both research and teaching. The study establishes feasibility rather than a scaling-efficiency claim, since single-node throughput used for comparison was estimated, not measured under matched conditions. Runbook and scripts are available (see Code Availability).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。