用文字指令动态控制对话轮换节奏，让机器人更懂对话氛围。
Prompt-Guided Turn-Taking Prediction
- 通过文本提示嵌入，动态调节对话轮换时机
- 在950小时数据上提升预测准确率，响应指令变化
- 适合需要灵活交互的对话系统研发者
对话轮换预测是语音对话系统和对话机器人的关键组件。近期方法采用基于Transformer的架构实现实时连续预测。本文提出一种新模型，通过文本提示实现对轮换行为的动态控制，使用户能以“更快”或“更平静”等指令直观调整对话节奏，适应不同对话伙伴与情境。该模型在基于Transformer的语音活动投影（VAP）基础上，将文本提示嵌入通道间Transformer及跨通道Transformer中。我们使用超过950小时的人类对话数据验证其可行性。由于现有数据集缺乏文本提示数据，我们利用大语言模型（LLM）生成合成提示语句。实验表明，该模型不仅提升了预测准确性，还能根据文本提示有效调节轮换时间行为。
原文摘要 · Abstract (English)
Turn-taking prediction models are essential components in spoken dialogue systems and conversational robots. Recent approaches leverage transformer-based architectures to predict speech activity continuously and in real-time. In this study, we propose a novel model that enables turn-taking prediction to be dynamically controlled via textual prompts. This approach allows intuitive and explicit control through instructions such as "faster" or "calmer" adapting dynamically to conversational partners and contexts. The proposed model builds upon a transformer-based voice activity projection (VAP) model, incorporating textual prompt embeddings into both channel-wise transformers and a cross-channel transformer. We evaluated the feasibility of our approach using over 950 hours of human-human spoken dialogue data. Since textual prompt data for the proposed approach was not available in existing datasets, we utilized a large language model (LLM) to generate synthetic prompt sentences. Experimental results demonstrated that the proposed model improved prediction accuracy and effectively varied turn-taking timing behaviors according to the textual prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。