提出轻量级适配器,让等变图网络高效适应新化学环境。
Magnitude-Modulated Equivariant Adapter for Parameter-Efficient Fine-Tuning of Equivariant Graph Neural Networks
- 用标量门控调节各阶特征幅度,保持对称性不变。
- 在多个基准上实现能量与力预测的最先进水平,参数更少。
- 适合需要保持物理对称性的分子模拟任务,如材料设计。
基于球谐函数的预训练等变图神经网络为计算昂贵的从头算方法提供了高效且准确的替代方案,但将其适配到新任务和化学环境仍需微调。传统参数高效微调(PEFT)技术如适配器和LoRA通常破坏对称性,与等变架构不兼容。近期提出的ELoRA是首个等变PEFT方法,提升了参数效率和性能。然而,其在每阶张量中保留的自由度仍可能扰动预训练特征分布,导致性能下降。为此,本文提出一种新型等变微调方法——幅度调制等变适配器(MMEA),通过轻量级标量门控在每阶和每多重性基础上调节特征幅度。实验表明,MMEA严格保持等变性,在多个基准上持续提升能量与力预测精度至当前最优水平,且训练参数少于现有方法。结果表明,在多数实际场景中,仅通过调节通道幅度即可适应等变模型至新化学环境而不破坏对称性,为等变PEFT设计开辟了新范式。
原文摘要 · Abstract (English)
Pretrained equivariant graph neural networks based on spherical harmonics offer efficient and accurate alternatives to computationally expensive ab-initio methods, yet adapting them to new tasks and chemical environments still requires fine-tuning. Conventional parameter-efficient fine-tuning (PEFT) techniques, such as Adapters and LoRA, typically break symmetry, making them incompatible with those equivariant architectures. ELoRA, recently proposed, is the first equivariant PEFT method. It achieves improved parameter efficiency and performance on many benchmarks. However, the relatively high degrees of freedom it retains within each tensor order can still perturb pretrained feature distributions and ultimately degrade performance. To address this, we present Magnitude-Modulated Equivariant Adapter (MMEA), a novel equivariant fine-tuning method which employs lightweight scalar gating to modulate feature magnitudes on a per-order and per-multiplicity basis. We demonstrate that MMEA preserves strict equivariance and, across multiple benchmarks, consistently improves energy and force predictions to state-of-the-art levels while training fewer parameters than competing approaches. These results suggest that, in many practical scenarios, modulating channel magnitudes is sufficient to adapt equivariant models to new chemical environments without breaking symmetry, pointing toward a new paradigm for equivariant PEFT design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。