用谱分布替代秩来衡量图模型容量,让容量可调可控。
Rank Is Not Capacity: Spectral Occupancy for Latent Graph Models

- 以归一化谱分布代替传统秩,量化模型真实学习能力。
- 通过单标量控制训练过程中的有效维度,实现精准容量调节。
- 适用于超参数调优困难的复杂网络建模,尤其适合高维场景。
图表示学习已成为分析网络数据的标准方法,隐变量嵌入广泛用于链接预测、社区发现等任务。然而,隐空间维度这一基础设计仍被视为脆弱的超参数,需在训练前固定并依赖预留性能调优。且学习到的因子仅在旋转与缩放下可识别,名义秩常不反映模型行为的真实维度。本文提出谱前缀提取与容量目标分析(Spectra),以学习到的正定核的谱分布取代秩作为分析单位,并通过迹归一化使不同拟合结果间的谱可比。归一化特征值构成单纯形上的分布,其香农有效秩既可总结学习容量,又可作为训练时的可控坐标:单一标量即可调控训练中实现的维度,二分法可精确达到设定的目标值。理论证明了实现维度曲线的局部光滑性与单调性。在合作、社交、生物及基础设施网络上,Spectra清晰刻画出性能-容量前沿,揭示预测精度与实际维度间的权衡。其表现媲美强基准,通过谱前缀可生成同一模型的低容量对齐视图,并在过参数化情形下提供容量的合理控制。容量因此成为拟合后模型的属性,而非训练超参数。
原文摘要 · Abstract (English)
Graph representation learning has become a standard approach for analyzing networked data, with latent embeddings widely used for link prediction, community detection, and related tasks. Yet a basic design choice, the latent dimension, is still treated as a brittle hyperparameter, fixed before training and tuned by held-out performance. Learned factors are also identifiable only up to rotation and rescaling, so the nominal rank rarely coincides with the quantity that governs model behavior. We propose Spectral Prefix Extraction and Capacity-Targeted Representation Analysis (Spectra), which replaces rank as the unit of analysis with the spectrum of a learned positive semidefinite kernel, trace-normalized so that spectra are comparable across fits. The normalized eigenvalues form a distribution on the simplex, and their Shannon effective rank acts both as a summary of learned capacity and as a controllable training-time coordinate: a single scalar shapes this realized dimension during training, and bisection targets any desired value within the rank cap. To theoretically support that, we show local regularity and monotonicity of the realized-dimension profile. Across collaboration, social, biological, and infrastructure networks, Spectra traces performance--capacity frontiers that make the trade-off between predictive accuracy and realized dimension visible. It performs competitively with strong link-prediction baselines, yields aligned lower-capacity views of the same fitted model through spectral prefixes, and provides a principled handle on capacity in the overparameterized regime. Capacity thus becomes a property of the fitted model rather than a hyperparameter of the training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。