用近似方法实现股票间注意力的高效建模，效果媲美完整计算。
Nystr\"om Attention Matches Full Attention for Cross-Sectional Stock Prediction
- 采用奈斯特罗姆低秩近似，以少量关键点模拟全连接注意力。
- 在300只股票时，近似结果与全注意力模型无显著差异（p<0.05）。
- 适合需要高效处理大量股票关系的量化投资研究者。
MASTER模型中的跨股票多头注意力模块占模型参数的42.5%和预测价值的25%。我们系统分析发现，该模块的注意力分布接近均匀（困惑度278/300），但强制完全均匀会丧失跨股票区分能力。谱分析揭示：偏离均匀性的部分为低秩结构（有效秩约65，前10个模态占能量96.5%），解释了为何稀疏近似始终失败，而奈斯特罗姆低秩注意力（取m=32个地标点）可在O(mN)成本下匹配O(N²)全注意力性能——通过等效性检验（TOST）在N=300（5次随机种子，Rank IC p=0.003）和N=800（10次随机种子，Rank IC p=0.034）均成立。进一步发现：(i) 注意力与收益相似性负相关（斯皮尔曼ρ = -0.614；行业子集下分别为-0.645与-0.627，控制行业、β和波动率后仍显著），表明其偏好互补而非相关性挖掘；(ii) 所有图结构替代方案均降低性能，硬掩码甚至劣于移除模块；(iii) 当股票数N≈3,500且采用适配架构时，任意跨股票模块（GCN、奈斯特罗姆或MASTER式流程）均未显著优于单股票LSTM基线（n=4）。结果表明，跨股票注意力的价值在于可压缩、动态、近乎全局的重分配机制，支持低秩近似但抗拒稀疏化。
原文摘要 · Abstract (English)
MASTER's inter-stock multi-head attention -- the module responsible for modeling cross-sectional stock relationships -- accounts for 42.5% of model parameters and 25% of predictive value. We systematically decompose this module and uncover a surprising structure: the learned attention is near-uniform (perplexity 278/300), yet forcing exact uniformity eliminates all cross-sectional discrimination. Spectral analysis resolves this paradox: the deviation from uniformity is low-rank (effective rank ~65, top-10 modes capture 96.5% of energy), explaining why sparse approximations consistently fail while Nystrom low-rank attention (m=32 landmarks) matches full O(N^2) attention at O(mN) cost -- certified equivalent via TOST at both N=300 (5 seeds, Rank IC p=0.003) and N=800 (10 seeds, Rank IC p=0.034). Additional findings include: (i) attention anti-correlates with return similarity (Spearman rho = -0.614; on the industry-labeled subset, -0.645 unconditionally and -0.627 after controlling for industry, beta, and volatility), suggesting complementarity-seeking rather than correlation mining; (ii) all graph-based alternatives degrade performance, with hard masking worse than complete module removal; and (iii) at N ~ 3,500 with adapted architectures, no cross-stock module (GCN, Nystrom, or MASTER-style pipeline) significantly outperforms a per-stock LSTM baseline (n=4 seeds), indicating that the benefits observed at smaller scales do not trivially transfer. These results establish that the inter-stock attention's value resides in a compressible, dynamic, near-global redistribution that rewards low-rank approximation but resists sparsification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。