提出可扩展的故障定位与恢复框架,解决大模型比特翻转问题。
BitFlipScope: Scalable Fault Localization and Recovery for Bit-Flip Corruptions in LLMs
- 通过输出、隐藏状态差异分析定位故障区域。
- 无参考模型时利用残差路径扰动识别受损部分。
- 无需微调即可轻量级恢复模型性能,适合高可靠性场景。
在实际和安全关键场景中部署的大语言模型(LLMs)越来越容易受到硬件退化、宇宙辐射或恶意攻击(如Rowhammer)导致的比特翻转故障影响。这些故障会无声地破坏内部参数,引发不可预测甚至危险的模型行为。定位故障至关重要:若无法确定受影响区域,则难以诊断退化根源、实施针对性修复,只能依赖昂贵的微调或全量重训。本文提出BitFlipScope,一种可扩展的软件框架,可在两种部署场景下识别Transformer架构中的故障区域。当存在干净参考模型时,通过输出、隐藏状态及内部激活的差异分析,检测异常行为以精确定位故障;当无参考模型时,利用残差路径扰动与损失敏感性分析,直接从受损模型推断故障影响区域。该框架不仅支持高效故障诊断,还可实现无需微调的轻量级性能恢复,为硬件脆弱及对抗性环境下的可信、容错式大模型部署提供可行路径。
原文摘要 · Abstract (English)
Large Language Models (LLMs) deployed in practical and safety-critical settings are increasingly susceptible to bit-flip faults caused by hardware degradation, cosmic radiation, or deliberate fault-injection attacks such as Rowhammer. These faults silently corrupt internal parameters and can lead to unpredictable or dangerous model behavior. Localizing these corruptions is essential: without identifying the affected region, it is impossible to diagnose the source of degradation, apply targeted corrective measures, or restore model functionality without resorting to costly fine-tuning or full retraining. This work introduces BitFlipScope, a scalable, software-based framework for identifying fault-affected regions within transformer architectures under two deployment scenarios. When a clean reference model is available, BitFlipScope performs differential analysis of outputs, hidden states, and internal activations for detecting anomalous behavior indicative of corruption to pinpoint or localize faults. When no reference model exists, it uses residual-path perturbation and loss-sensitivity profiling to infer the fault-impacted region directly from the corrupted model. In both settings, the framework not only enables effective fault diagnosis but also supports lightweight performance recovery without fine-tuning, offering a practical path to restoring corrupted models. Together, these capabilities make BitFlipScope an important step toward trustworthy, fault-resilient LLM deployment in hardware-prone and adversarial environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。