用中间层嵌入提升分子模型性能，平均增益5.4%。
Superior Molecular Representations from Intermediate Encoder Layers
- 选取最优中间层嵌入替代最终层，保留更通用特征。
- 22个任务上平均性能提升5.4%，最高达28.6%。
- 截断训练中间层，最高提升40.8%，适合高效建模场景。
预训练分子编码器在性质预测和分子生成等任务中已不可或缺。然而，仅依赖最终层嵌入可能丢失重要信息。本文分析五种不同分子编码器的信息流动，发现中间层保留更多通用特征，而最终层则专注压缩。在22个性质预测任务上进行逐层评估，结果表明：使用冻结的最优中间层嵌入，下游性能平均提升5.4%，最高达28.6%；若对编码器在中间深度截断并微调，平均提升达8.5%，最高提升40.8%，并在多个基准上取得新最佳结果。研究强调充分挖掘分子编码器的表征深度可显著提升性能与计算效率。代码将公开。
原文摘要 · Abstract (English)
Pretrained molecular encoders have become indispensable in computational chemistry for tasks such as property prediction and molecular generation. However, the standard practice of relying solely on final-layer embeddings for downstream tasks may discard valuable information. In this work, we first analyze the information flow in five diverse molecular encoders and find that intermediate layers retain more general-purpose features, whereas the final-layer specializes and compresses information. We then perform an empirical layer-wise evaluation across 22 property prediction tasks. We find that using frozen embeddings from optimal intermediate layers improves downstream performance by an average of 5.4%, up to 28.6%, compared to the final-layer. Furthermore, finetuning encoders truncated at intermediate depths achieves even greater average improvements of 8.5%, with increases as high as 40.8%, obtaining new state-of-the-art results on several benchmarks. These findings highlight the importance of exploring the full representational depth of molecular encoders to achieve substantial performance improvements and computational efficiency. The code will be made publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。