arXiv:2412.06249cs.CLcs.LG2024-12被引 15

通过多任务学习提升大模型性能,实现文本分类与摘要生成双优。

Optimizing Multi-Task Learning for Enhanced Performance in Large Language Models

  • 共享特征提取+任务专用模块,实现多任务知识共享与协同优化。
  • 在GLUE数据集上优于GPT-4、BERT等模型,分类准确率与摘要ROUGE值均更高。
  • 适合追求模型泛化能力与跨任务平衡的AI研发人员参考。

本研究探索基于GPT-4的多任务学习框架对大语言模型性能的提升方法,实验涵盖文本分类与自动摘要生成两个任务。通过共享特征提取器与任务专用模块的联合设计,实现同一模型中多个任务的知识共享与优化。实验采用GLUE数据集的多个子任务,对比了多任务模型与单任务GPT-4、多任务GPT-3、BERT基础模型及经典Bi-LSTM+Attention模型的表现。结果表明,所提多任务学习模型在文本分类准确率和摘要生成的ROUGE值上均优于其他对比模型,验证了多任务学习在提升模型泛化能力和任务间协同学习方面的优势。模型训练过程中保持稳定的损失收敛速率,展现出良好的学习效率与测试集适应性。研究证实了多任务学习框架在大语言模型中的适用性,尤其在平衡不同任务能力方面表现突出。未来结合大模型与多模态数据、动态任务调整技术,该框架有望在更广泛领域发挥作用,为通用人工智能发展提供新思路。

原文摘要 · Abstract (English)

This study aims to explore the performance improvement method of large language models based on GPT-4 under the multi-task learning framework and conducts experiments on two tasks: text classification and automatic summary generation. Through the combined design of shared feature extractors and task-specific modules, we achieve knowledge-sharing and optimization of multiple tasks in the same model. The experiment uses multiple subtasks of the GLUE dataset to compare the performance of the multi-task model with the single-task GPT-4, the multi-task version of GPT-3, the BERT basic model, and the classic Bi-LSTM with Attention model. The results show that the proposed multi-task learning model outperforms other comparison models in terms of text classification accuracy and ROUGE value of summary generation, demonstrating the advantages of multi-task learning in improving model generalization ability and collaborative learning between tasks. The model maintains a stable loss convergence rate during training, showing good learning efficiency and adaptability to the test set. This study verifies the applicability of the multi-task learning framework in large language models, especially in improving the model's ability to balance different tasks. In the future, with the combination of large language models and multimodal data and the application of dynamic task adjustment technology, the framework based on multi-task learning is expected to play a greater role in practical applications across fields and provide new ideas for the development of general artificial intelligence.

多任务学习大模型优化GPT-4文本生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。