提出轻量级审计方法，精准定位序列模型中因果性失效的层。
The Mask Is Not the Model: Auditing Prefix Invariance in Attention, State-Space, and Hybrid Sequence Models
- 通过两次前向传播实现无训练、无梯度的因果性审计。
- 在192次注入故障测试中，准确定位所有因果性泄漏的层。
- 适用于注意力、状态空间及混合序列模型，适合模型开发者验证可靠性。
混合序列模型必须满足前缀不变性：位置t的表示不应依赖未来输入，但这一性质极少被验证。本文形式化定义前缀不变性，并提出一种轻量级审计方法，仅需两次前向传播，无需训练或梯度，即可生成每层得分以定位因果性破坏点。当前主流的注意力掩码检查不完整：因果性是图级别的属性，泄露可能通过扫描、聚合或归一化发生，即使掩码正确。在8个检查点上进行192次注入故障测试，掩码检查未发现任何问题，而本方法成功定位全部192处泄漏。对Transformer分块扫描代码的静态/动态分析发现Zamba2和Nemotron-H存在相同缺陷（跨块轴错误），已通过参考实现修复。该方法可一页呈现，运行仅需数秒。
原文摘要 · Abstract (English)
Hybrid sequence models must satisfy prefix invariance: representations at position t must not depend on future inputs, yet this is rarely verified. We formalize prefix invariance and give a lightweight audit, two forward passes, no training or gradients, yielding a per-layer score localizing where causality breaks. Attention-mask inspection, the field's default check, is incomplete: causality is a graph-level property, and leaks can occur via scans, aggregations, or normalization despite correct masks. Across 192 injected-fault trials on eight checkpoints, mask inspection detected none, while our audit localized all 192/192 to the exact layer. Static/dynamic analysis of chunked-scan code in transformers found the same defect in Zamba2 and Nemotron-H, an inter-chunk axis error fixed via the reference implementation. The method fits on one page and runs in seconds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。