让微服务重启更安全,避免连锁故障。
Rebooting Microreboot: Architectural Support for Safe, Parallel Recovery in Microservice Systems
- 分诊断、规划、验证三阶段,用指令集定义安全重启动作
- 在线推导最小重启组,99%场景下耗时仅21毫秒
- 实测可降低95%误操作伤害,适合高可靠系统
微重启通过只重启故障组件实现快速恢复,但在现代微服务中,密集依赖导致随意重启可能引发连锁故障。自主修复代理在无安全保障下执行底层指令,风险更高。本文提出分离规划与执行的三代理架构(诊断、规划、验证),基于包含七种操作的类型化指令集(ISA)制定带副作用语义的修复方案,并由小型微内核事务式验证与执行。代理均视为不可信,安全性来自ISA和微内核。通过分布式追踪在线推断恢复边界,计算最小重启组与顺序约束。在阿里、Meta工业级追踪数据及DeathStarBench注入故障测试中,恢复组推断在P99下耗时21毫秒;类型化执行使仿真中代理引发的损害减少95%,线上实现0%损害。核心价值在于安全性而非速度:大模型推理开销反而延长了具备快速自动重启能力的服务的恢复时间(TTR)。
原文摘要 · Abstract (English)
Microreboot enables fast recovery by restarting only the failing component, but in modern microservices naive restarts are unsafe: dense dependencies mean rebooting one service can disrupt many callers. Autonomous remediation agents compound this by actuating raw infrastructure commands without safety guarantees. We make microreboot practical by separating planning from actuation: a three-agent architecture (diagnosis, planning, verification) proposes typed remediation plans over a seven-action ISA with explicit side-effect semantics, and a small microkernel validates and executes each plan transactionally. Agents are explicitly untrusted; safety derives from the ISA and microkernel. To determine where restart is safe, we infer recovery boundaries online from distributed traces, computing minimal restart groups and ordering constraints. On industrial traces (Alibaba, Meta) and DeathStarBench with fault injection, recovery-group inference runs in 21 ms at P99; typed actuation reduces agent-caused harm by 95% in simulation and achieves 0% harm online. The primary value is safety, not speed: LLM inference overhead increases TTR for services with fast auto-restart.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。