发现更丰富的分子特征未必提升泛化能力,挑战自监督学习常见假设。
Spectral Analysis of Molecular Features: When Richer Features Do Not Guarantee Better Generalization
- 通过谱分析对比多种分子表示在核岭回归中的表现
- 局部3D特征谱越丰富,泛化性能反而越差,仅需不到2%特征值保留95%性能
- 揭示任务与表征类型对模型泛化的影响,为科学小样本学习提供新视角
特征嵌入的谱特性对模型泛化与表征质量具有关键启示。尽管深度学习广泛用于分子性质预测,核方法在低数据场景仍具竞争力,但其谱行为尚未被深入研究。本文首次系统分析了核岭回归在多种表征(包括分子指纹ECFP、预训练变换器、图神经网络和3D描述符)下的谱特性,涵盖QM9和3 MoleculeNet基准。出人意料的是,更丰富的谱特征并不总带来更好泛化性能,与自监督学习中普遍使用的表征启发式相悖。在4种谱度量中,仅ECFP基核显示性能与谱正相关;变换器和全局3D表示表现混杂,而局部3D表示则始终呈负相关。截断分析进一步揭示:在热力学目标上,局部3D表示仅需少于2%的特征值(有时低至0.02%)即可恢复95%性能,而ECFP和变换器核则需要更多。结果表明,泛化性能强烈依赖任务与表征类型,挑战了‘更丰富谱即更优’的常识,为自监督学习及标签受限科学任务中的表征评估提供新指导。
原文摘要 · Abstract (English)
The spectral properties of feature embeddings offer critical insights into model generalization and representation quality. While deep learning models are widely used for molecular property prediction, kernel methods remain competitive in low-data regimes, yet their spectral behavior is largely unexplored. We present the first comprehensive spectral analysis of kernel ridge regression across diverse representations-including molecular fingerprints (ECFP), pretrained transformers, graph neural networks, and 3D descriptors-evaluated on QM9 and 3 MoleculeNet benchmarks. Surprisingly, richer spectral features do not consistently yield better generalization performance, contradicting common representation heuristics used in self-supervised learning (SSL). Across 4 spectral metrics, only ECFP-based kernels show a strictly positive correlation with performance. Transformer and global 3D representations exhibit mixed behavior, whereas local 3D representations show consistently negative correlations. Truncation analysis further emphasizes this disparity: for local 3D representations on thermodynamic targets, fewer than 2\% of eigenvalues (and occasionally as few as 0.02\%) are needed to recover 95\% of performance, whereas ECFP and transformer kernels require significantly more. By demonstrating a strong dependence on both task and representation, our results challenge the heuristic that richer spectra inherently improve generalization, providing new guidance for evaluating representations in SSL and in label-limited scientific tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。