解决AI训练在macOS上检查点损坏问题，提升可靠性。
Crash-Consistent Checkpointing for AI Training on macOS/APFS
- 设计三种写入模式，逐步增强文件与目录的持久化保障。
- 通过哈希校验与自动回滚，检测99.8%以上数据损坏且无误报。
- 适合重视训练可靠性、能接受一定性能开销的AI研发团队。
深度学习训练依赖周期性检查点以应对故障，但不安全的检查点安装可能导致磁盘文件损坏。本文针对macOS/APFS系统开展实验研究，实现三种递增耐用性保障的写入模式：不安全（基线，无fsync）、atomic_nodirsync（文件级通过fsync保障）、atomic_dirsync（文件+目录级保障）。设计一种无需格式感知的完整性保护机制，使用SHA-256校验并支持自动回滚。通过受控实验，包括430次崩溃注入测试和1600次数据损坏注入测试，验证该机制可检测99.8%-100%的损坏，且零误报。性能开销方面，atomic_nodirsync为基线的56.5%-108.4%，atomic_dirsync高达84.2%-570.6%。研究量化了可靠性与性能的权衡，为生产环境部署提供指导。
原文摘要 · Abstract (English)
Deep learning training relies on periodic checkpoints to recover from failures, but unsafe checkpoint installation can leave corrupted files on disk. This paper presents an experimental study of checkpoint installation protocols and integrity validation for AI training on macOS/APFS. We implement three write modes with increasing durability guarantees: unsafe (baseline, no fsync), atomic_nodirsync (file-level durability via fsync()), and atomic_dirsync (file + directory durability). We design a format-agnostic integrity guard using SHA-256 checksums with automatic rollback. Through controlled experiments including crash injection (430 unsafe-mode trials) and corruption injection (1,600 atomic-mode trials), we demonstrate that the integrity guard detects 99.8-100% of corruptions with zero false positives. Performance overhead is 56.5-108.4% for atomic_nodirsync and 84.2-570.6% for atomic_dirsync relative to the unsafe baseline. Our findings quantify the reliability-performance trade-offs and provide deployment guidance for production AI infrastructure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。