用复数状态传播实现确定性序列任务的精准跟踪
State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking
- 仅通过复数域中的状态旋转传递信息，无中间输出投影
- 在奇偶校验等任务上达100%准确率与完美F1分数
- 适合需要高效精确状态追踪的场景，如形式验证
基于Transformer的架构在序列建模中占据主导地位，主要得益于注意力机制的强大表达能力。然而，对于一类确定性状态追踪任务——如奇偶校验、模计数和括号匹配——注意力机制可能过度复杂。本文表明，仅靠状态传播就已足够。我们提出复数状态传播器（CSP），一种极简的递归架构，仅在层间传播隐藏状态，不进行中间步骤的输出投影。状态以复数向量表示，通过输入相关的复数域旋转进行更新。为实现深层传播且避免梯度消失或退化，引入块级跳连，并在序列边界采用逐元素复数归一化与SiLU激活。结合焦点损失，CSP在典型任务上实现了100%准确率与完美F1分数。
原文摘要 · Abstract (English)
Transformer-based architectures have dominated sequence modeling, largely due to the expressive power of attention mechanisms. However, for a class of deterministic state tracking tasks---such as parity checking, modular counting, and parenthesis matching---attention may be overkill. In this paper, we show that \textbf{state propagation alone is sufficient}. We propose the \textbf{Complex State Propagator (CSP)}, a minimalistic recurrent architecture that \textbf{only propagates hidden states} across layers without output projections at intermediate steps. The state is represented as a complex-valued vector, updated via input-dependent rotations in the complex domain. To enable deep propagation without gradient vanishing or degradation, we introduce a \textbf{block-level skip connection} alongside element-wise complex normalization and SiLU activation at sequence boundaries. Applied with Focal Loss, CSP achieves \textbf{100\% accuracy} with perfect F1 scores across canonical tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。