通过多任务学习提升大模型在电商领域的适应能力。
More diverse more adaptive: Comprehensive Multi-task Learning for Improved LLM Domain Adaptation in E-commerce
- 设计电商多任务框架，逐步引入新能力任务和子任务。
- 实验证明任务多样性显著提升模型性能，且与模型规模协同增益。
- 成果在KDD Cup 2024中排名第五，适合电商场景的模型优化研究者。
近年来，大语言模型（LLMs）因其强大的领域适应能力被广泛应用于多个领域。已有研究指出，多样化的多模态数据可增强LLMs的领域适应性能，但该假设在电商领域尚未充分验证。为此，本文提出一个全面的电商多任务学习框架，并设计实证实验，从“能力全面性”和“任务全面性”两个角度考察多样数据与任务对LLM的影响。具体而言，通过逐步引入涉及新主要能力领域的任务，以及持续添加不同能力域内的子任务，观察到LLM性能显著提升。此外，发现增加模型容量能放大多样性的收益，表明模型能力与数据多样性存在协同效应。最后，将实验中表现最佳的模型应用于KDD Cup 2024，Task 1中取得第五名成绩。该结果验证了本研究在推动大模型于电商领域应用中的重要意义。
原文摘要 · Abstract (English)
In recent years, Large Language Models (LLMs) have been widely applied across various domains due to their powerful domain adaptation capabilities. Previous studies have suggested that diverse, multi-modal data can enhance LLMs' domain adaptation performance. However, this hypothesis remains insufficiently validated in the e-commerce sector. To address this gap, we propose a comprehensive e-commerce multi-task framework and design empirical experiments to examine the impact of diverse data and tasks on LLMs from two perspectives: "capability comprehensiveness" and "task comprehensiveness." Specifically, we observe significant improvements in LLM performance by progressively introducing tasks related to new major capability areas and by continuously adding subtasks within different major capability domains. Furthermore, we observe that increasing model capacity amplifies the benefits of diversity, suggesting a synergistic relationship between model capacity and data diversity. Finally, we validate the best-performing model from our empirical experiments in the KDD Cup 2024, achieving a rank 5 in Task 1. This outcome demonstrates the significance of our research for advancing LLMs in the e-commerce domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。