动态调整提示词组合,提升大模型多任务跨域适应能力
Dynamic Prompt Fusion for Multi-Task and Cross-Domain Adaptation in LLMs
- 用可变提示池和任务感知调度动态融合提示
- 在多个语言理解任务上准确率提升显著,稳定性强
- 适合需要统一建模多任务的场景,如跨领域应用
本研究针对大语言模型在多任务与跨领域设置下的泛化能力不足问题,提出一种统一的多任务学习框架,包含动态提示调度机制。通过引入提示池与任务感知调度策略,模型能动态组合并对齐不同任务的提示,增强捕捉任务间语义差异的能力。在提示融合过程中,利用任务嵌入与门控机制精细控制提示信号,确保内容与任务需求一致,并构建灵活的任务间共享路径。优化目标聚焦联合多任务学习,采用自动学习的调度权重策略,有效缓解任务干扰与负迁移。敏感性实验验证了提示温度参数与任务数量变化的影响,结果表明该机制显著提升模型稳定性与迁移能力。实验显示,该方法在多项语言理解与知识推理任务中表现优异,充分证明其在统一多任务建模与跨域适应中的有效性。
原文摘要 · Abstract (English)
This study addresses the generalization limitations commonly observed in large language models under multi-task and cross-domain settings. Unlike prior methods such as SPoT, which depends on fixed prompt templates, our study introduces a unified multi-task learning framework with dynamic prompt scheduling mechanism. By introducing a prompt pool and a task-aware scheduling strategy, the method dynamically combines and aligns prompts for different tasks. This enhances the model's ability to capture semantic differences across tasks. During prompt fusion, the model uses task embeddings and a gating mechanism to finely control the prompt signals. This ensures alignment between prompt content and task-specific demands. At the same time, it builds flexible sharing pathways across tasks. In addition, the proposed optimization objective centers on joint multi-task learning. It incorporates an automatic learning strategy for scheduling weights, which effectively mitigates task interference and negative transfer. To evaluate the effectiveness of the method, a series of sensitivity experiments were conducted. These experiments examined the impact of prompt temperature parameters and task number variation. The results confirm the advantages of the proposed mechanism in maintaining model stability and enhancing transferability. Experimental findings show that the prompt scheduling method significantly improves performance on a range of language understanding and knowledge reasoning tasks. These results fully demonstrate its applicability and effectiveness in unified multi-task modeling and cross-domain adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。