用轻量模块组合构建复合人格，无需再训练。
Domain Expansion: Parameter-Efficient Modules as Building Blocks for Composite Domains
- 在权重空间直接组合已微调的轻量模块
- 16种MBTI人格由8个特质模块组合而成
- 适合需要快速适配新场景的轻量化应用
参数高效微调（PEFT）已成为大规模预训练模型微调的高效替代方案。随着预训练模型规模持续扩大，可利用PEFT训练一系列参数高效的模块（PEMs），使其成为不同领域的专家。本文探索将这些独立微调的PEMs进行组合，以实现对复合领域分布的泛化能力。通过仅在权重空间操作的简单组合函数实现模块融合，无需额外微调。该方法应用于16种迈尔斯-布里格斯类型指标（MBTI）复合人格的建模，基于4个基础二元维度构成的8个个体特质模块，可合并生成唯一人格。通过在线MBTI人格测试问卷评估了单个特质模块与组合后的人格模块，验证了PEFT在训练PEMs及无须再训练即可实现模块组合的有效性。
原文摘要 · Abstract (English)
Parameter-Efficient Fine-Tuning (PEFT) is an efficient alternative to full scale fine-tuning, gaining popularity recently. With pre-trained model sizes growing exponentially, PEFT can be effectively utilized to fine-tune compact modules, Parameter-Efficient Modules (PEMs), trained to be domain experts over diverse domains. In this project, we explore composing such individually fine-tuned PEMs for distribution generalization over the composite domain. To compose PEMs, simple composing functions are used that operate purely on the weight space of the individually fine-tuned PEMs, without requiring any additional fine-tuning. The proposed method is applied to the task of representing the 16 Myers-Briggs Type Indicator (MBTI) composite personalities via 4 building block dichotomies, comprising of 8 individual traits which can be merged (composed) to yield a unique personality. We evaluate the individual trait PEMs and the composed personality PEMs via an online MBTI personality quiz questionnaire, validating the efficacy of PEFT to fine-tune PEMs and merging PEMs without further fine-tuning for domain composition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。