用模拟对话生成真实用户问题，提升广告领域问答质量
DeepThink: Aligning Language Models with Domain-Specific User Intents
- 通过种子问题模拟真实用户提问，再以对话挖掘隐藏需求
- 相比GPT-4-turbo+RAG，综合性能提升7.92%（含相关性、准确度等）
- 适合需要精准理解领域用户意图的智能客服与问答系统
监督微调中合成指令常偏离真实用户问题和期望答案。本文提出DeepThink框架：先生成少量种子问题模拟真实提问，再通过模拟对话揭示潜在用户需求，并结合上下文与检索文档优化回答，提升全面性。实验表明，在广告领域的实际用户测试集上，DeepThink相较GPT-4-turbo+RAG助手在相关性、完整性、清晰度、准确性和可操作性等维度平均提升7.92%。
原文摘要 · Abstract (English)
Supervised fine-tuning with synthesized instructions has been a common practice for adapting LLMs to domain-specific QA tasks. However, the synthesized instructions deviate from real user questions and expected answers. This study proposes a novel framework called DeepThink to generate high-quality instructions. DeepThink first generates a few seed questions to mimic actual user questions, simulates conversations to uncover the hidden user needs, and refines the answer by conversational contexts and the retrieved documents for more comprehensive answers. Experiments demonstrate that DeepThink achieves an average performance improvement of 7.92% compared to a GPT-4-turbo+RAG-based assistant on the real user test set in the advertising domain across dimensions such as relevance, completeness, clarity, accuracy, and actionability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。