arXiv:2510.12111cs.LGcs.AI2025-10被引 2

Chimera用状态空间模型统一建模序列、图像和图数据,无需位置编码等特定偏置。

Chimera: State Space Models Beyond Sequences

  • 将状态空间模型扩展到任意图结构,直接利用数据拓扑信息
  • 在GLUE、ImageNet-1k和长程图任务上均优于BERT、ViT等基线
  • 支持线性时间计算(有向无环图)和二次复杂度(一般图),无需领域启发式

基于Transformer的深度学习方法已成为建模序列、图像和图数据的标准范式。这些方法依赖自注意力机制,将数据视为无序元素集合,忽略其邻域结构或图拓扑,需引入归纳偏置(如序列和图像的位置编码,或图上的随机游走)来融入拓扑信息。然而,设计这类任务特异性偏置需要大量工作,并可能引发副作用,阻碍泛化能力。本文提出Chimera,一种以严谨方式直接整合数据拓扑的统一模型,消除对领域特定偏置的需求。核心思想是:状态空间模型天然无需位置编码,可推广至捕捉任意图拓扑。实验表明,Chimera在语言、视觉和图领域均表现强劲,相较BERT在GLUE上提升0.7点,相较于ViT在ImageNet-1k上提升2.6%,在长程图基准(Long Range Graph Benchmark)上优于所有基线。我们进一步提出算法优化:(1) 对有向无环图,Chimera可实现线性时间递推;(2) 对一般图,通过简单数学松弛即可达到Transformer的二次复杂度,无需领域特定启发式。这些结果验证了Chimera的核心贡献,支持数据拓扑作为跨模态的强大归纳偏置。

原文摘要 · Abstract (English)

Transformer-based deep learning methods have become the standard approach for modeling diverse data such as sequences, images, and graphs. These methods rely on self-attention, which treats data as an unordered set of elements. This ignores the neighborhood structure or graph topology of the data and requires inductive biases--such as position embeddings in sequences and images, or random walks in graphs--to incorporate topology. However, designing such task-specific biases requires significant effort and can introduce side effects that hinder generalization. We introduce Chimera, a unified model that directly incorporates data topology in a principled way, removing the need for domain-specific biases. The key idea is that state space models--which naturally do not require position embeddings--can be generalized to capture any graph topology. Our experiments show that Chimera achieves strong performance across language, vision, and graph domains, outperforming BERT on GLUE by 0.7 points, ViT on ImageNet-1k by 2.6%, and all baselines on the Long Range Graph Benchmark. We further propose algorithmic optimizations to improve Chimera's efficiency: (1) for Directed Acyclic Graphs, Chimera can be implemented as a linear-time recurrence; (2) for general graphs, a simple mathematical relaxation achieves Transformer's quadratic complexity without domain-specific heuristics. These results validate Chimera's core contribution and support the idea that data topology is a powerful inductive bias across modalities.

状态空间模型图神经网络统一建模拓扑感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。