从直播视频中自动挖掘高质量客服对话数据,解决垂直领域语料稀缺难题。
STREAM: A Data-Centric Framework for Mining High-Value Task-Oriented Dialogues from Streaming Media

- 利用直播和短视频中的真实互动信号,结合角色设定与对话蓝图生成对话。
- 构建了含8.7万会话、150万轮次的StreamDial数据集,平均每个会话17.11轮。
- 适合需要高真实感服务对话训练的模型开发者,尤其关注多领域对话系统。
垂直领域大语言模型受限于复杂、特定领域的任务导向对话数据稀缺。现有数据获取方式面临三重困境:专家标注成本高,真实服务对话受隐私与商业限制,静态语料迅速过时。本文提出Stream框架,通过挖掘公开流媒体(直播与短视频)中的真实交互信号,规模化合成高质量服务对话。Stream结合角色化人物设定与对话蓝图构建,并采用检索增强生成(RAG)实现知识感知回复。基于此,我们发布StreamDial,一个覆盖汽车、餐饮、酒店的多领域大规模数据集,共包含87,498个对话会话和1,497,320轮对话,平均每会话17.11轮,各领域规模均衡。每个会话以结构化四元组⟨P_u, P_a, B, H⟩组织,包含用户/代理角色、对话蓝图与历史记录,捕捉需求挖掘、约束冲突、协商与恢复等真实服务行为。自动评估与下游任务验证表明,StreamDial显著提升对话内在质量,基于该数据训练的模型在多种骨干网络上均提升对话状态跟踪性能;我们还完成了人工评估,并在控制训练预算下报告Qwen3-8B模型的有前景多语言迁移结果。数据已开源:https://github.com/hitxueliang/DialogDataSetBySTREAM。
原文摘要 · Abstract (English)
Large language models for vertical domains are bottlenecked by the scarcity of complex, domain-specific task-oriented dialogues. Existing data acquisition pipelines face a persistent trilemma: expert annotation is expensive, real-world service conversations are constrained by privacy and commercial restrictions, and static corpora quickly become temporally stale. We propose Stream, a data-centric framework that leverages publicly available streaming media (live streams and short videos) to synthesize high-value service dialogues at scale. Stream mines authentic interaction signals from noisy streams and synthesizes conversations by integrating role-grounded persona construction with Conversational Blueprint construction; it further adopts retrieval-augmented generation (RAG) to support knowledge-aware responses. Based on Stream, we release StreamDial, a large-scale multi-domain dataset covering Automotive, Restaurant, and Hotel. StreamDial contains 87,498 dialogue sessions and 1,497,320 turns in total, with an average of 17.11 turns per session and a comparable scale across domains. Each session is organized as a structured quadruplet $\langle P_u, P_a, B, H \rangle$ that pairs dialogue history with explicit user/agent personas and a Conversational Blueprint, capturing realistic service behaviors such as requirement mining, constraint conflicts, negotiation, and recovery. Evaluations with automatic judges and downstream tasks show that StreamDial improves intrinsic dialogue quality over strong baselines, and models trained with StreamDial improve Dialogue State Tracking across backbones; we further report a completed human-evaluation set and encouraging multilingual transfer on Qwen3-8B under a controlled training budget. The data is released in https://github.com/hitxueliang/DialogDataSetBySTREAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。