arXiv:2605.14727cs.CV2026-05

提出跨频段对齐的谱令牌混合器,提升视觉特征建模效率。

CHASM: Cross-frequency Harmonized Axis-Separable Mixing for Spectral Token Operators

论文配图:CHASM: Cross-frequency Harmonized Axis-Separable Mixing for Spectral Token Operators
图 1 · 摘自论文原文
  • 共享通道特征基底,保留各频段自适应增益
  • 在加速磁共振重建等任务中性能超越基线
  • 适合需要高效全局交互的视觉模型改进

基于傅里叶变换的谱令牌混合器能高效建模视觉特征图中的全局交互。现有方法或沿固定通道轴应用滤波器级谱响应,或学习自适应频率索引通道混合,但未显式对齐不同频段间的通道方向。我们提出 CHASM——一种跨频段对齐的轴分离混合器,作为结构化的折中方案。该方法将应共享的部分与应保留频段特性的部分分离:所有频段共享一个学习得到的通道特征基底,而每个频段保留自身的正向谱增益。共享基底使不同频段的通道方向可比,正增益则保持局部谱自适应性。CHASM 将此结构化算子分别应用于高度和宽度轴,并可作为即插即用模块嵌入现有骨干网络。我们提供了共享基底算子族的结构性分析,并通过同骨干对比评估了 CHASM。在加速磁共振重建、欠采样磁共振分割和自然图像重建任务中,CHASM 均一致优于同骨干的谱混合器基线。消融实验表明,移除共享基底约束会降低性能,随机化一致采样几何显著削弱增益,验证了跨频段对齐作为谱令牌算子的有效归纳偏置。

原文摘要 · Abstract (English)

Spectral token mixers based on Fourier transforms provide an efficient way to model global interactions in visual feature maps. Existing designs often either apply filter-wise spectral responses along fixed channel axes, or learn adaptive frequency-indexed channel mixing without explicitly aligning the channel directions used across frequencies. We propose CHASM, a Cross-frequency Harmonized Axis-Separable Mixer, as a structured middle ground. CHASM separates what should be shared from what should remain frequency-specific: all frequencies share a learned channel eigenbasis, while each frequency retains its own positive spectral gains. The shared basis makes channel directions comparable across the spectrum, whereas the positive gains preserve local spectral adaptivity. CHASM applies this structured operator separably along the height and width axes and is used as a drop-in replacement mixer inside existing backbones. We provide a structural characterization of the shared-basis operator family and evaluate CHASM through controlled same-backbone comparisons. Across accelerated MRI reconstruction, undersampled MRI segmentation, and natural-image reconstruction, CHASM consistently improves over same-backbone spectral-mixer baselines. Ablations show that removing the shared-basis constraint weakens performance, and randomizing coherent sampling geometry substantially reduces the gain, supporting cross-frequency harmonization as a useful inductive bias for spectral token operators.

谱混合器视觉建模傅里叶变换MRI重建

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。