通过渐增任务提升小数据分子预测性能，计算开销低
Task Addition in Multi-Task Learning by Geometrical Alignment
- 先在大数据集上预训练，再逐个添加目标任务模块
- 在有限数据下表现优于传统多任务方法
- 适合小样本分子属性预测场景
在数据有限的情况下训练深度学习模型并保持泛化能力，是分子属性预测中的核心挑战。一种有效方法是从大规模数据集中迁移知识。近期提出的几何对齐迁移编码器（GATE）通过任务特定隐空间的几何形状对齐实现软参数共享，但其在扩展至多任务时面临计算成本高的问题。本文提出一种GATE的任务增量方法：先在大型数据集上进行监督多任务预训练，随后为每个目标任务添加并训练专属模块。实验表明，该策略在保持相近计算成本的前提下，显著提升了目标任务在小样本数据上的性能。
原文摘要 · Abstract (English)
Training deep learning models on limited data while maintaining generalization is one of the fundamental challenges in molecular property prediction. One effective solution is transferring knowledge extracted from abundant datasets to those with scarce data. Recently, a novel algorithm called Geometrically Aligned Transfer Encoder (GATE) has been introduced, which uses soft parameter sharing by aligning the geometrical shapes of task-specific latent spaces. However, GATE faces limitations in scaling to multiple tasks due to computational costs. In this study, we propose a task addition approach for GATE to improve performance on target tasks with limited data while minimizing computational complexity. It is achieved through supervised multi-task pre-training on a large dataset, followed by the addition and training of task-specific modules for each target task. Our experiments demonstrate the superior performance of the task addition strategy for GATE over conventional multi-task methods, with comparable computational costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。