让大模型自己生成对话来提升摘要能力，实现少样本摘要效果更好
Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarization
- 用摘要能力指导对话生成，再用生成的对话-摘要对优化摘要能力
- 在少样本场景下ROUGE提升1.5%，BERTScore提升0.3%
- 适合想提升少样本摘要性能的研究者和开发者
本文提出一种基于大模型内部知识的相互增强数据合成方法（MRDS），用于提升少样本对话摘要性能。不同于依赖外部知识的方法，该方法通过双向强化模型的对话生成与摘要能力，在训练中相互促进。其中，摘要能力通过生成高质量的对话-摘要配对数据来增强；对话生成能力则通过摘要评分进行定向偏好优化。利用该机制，我们从模型内部提取合成数据，补充真实少样本训练集。实验表明，该方法在少样本设置下使ROUGE得分提升1.5%，BERTScore提升0.3%。此外，在人类评估中平均得分最高，优于预训练模型及仅针对摘要任务微调的基线。
原文摘要 · Abstract (English)
In this work, we propose Mutual Reinforcing Data Synthesis (MRDS) within LLMs to improve few-shot dialogue summarization task. Unlike prior methods that require external knowledge, we mutually reinforce the LLMś dialogue synthesis and summarization capabilities, allowing them to complement each other during training and enhance overall performances. The dialogue synthesis capability is enhanced by directed preference optimization with preference scoring from summarization capability. The summarization capability is enhanced by the additional high quality dialogue-summary paired data produced by the dialogue synthesis capability. By leveraging the proposed MRDS mechanism, we elicit the internal knowledge of LLM in the format of synthetic data, and use it to augment the few-shot real training dataset. Empirical results demonstrate that our method improves dialogue summarization, achieving a 1.5% increase in ROUGE scores and a 0.3% improvement in BERT scores in few-shot settings. Furthermore, our method attains the highest average scores in human evaluations, surpassing both the pre-trained models and the baselines fine-tuned solely for summarization tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。