让大模型学会识别、分类和理解笑声的社交含义
SMILE-Next: Teaching Large Language Models to Detect, Classify, and Reason about Laughter
- 用自生成笑声指令提升模型泛化能力
- 多专家路由机制动态选型，任务表现更优
- 首个涵盖三类任务的多模态笑声数据集
笑声是一种复杂的社交信号，其传达的意图远超单纯的欢乐。现有研究多聚焦于孤立的笑声分析任务，对真实场景中笑声的全面理解仍不充分。为此，我们提出SMILE-Next，一个包含多模态文本表示与问答标注的真实世界笑声理解数据集，覆盖笑声检测、笑声类型分类和笑声推理三个任务。基于该数据集，我们构建了一个专注笑声理解的大语言模型，提出两个核心组件：笑声专用Self-Instruct，通过自动生成多样化笑声相关指令增强跨任务与跨域泛化；以及多笑声专家混合（MoLE）框架，采用任务自适应专家路由机制，动态选择针对特定任务的专精专家，显著提升任务性能与效率。实验表明，所提方法显著优于多模态大模型基线，推动了真实场景下笑声理解的进展。
原文摘要 · Abstract (English)
Laughter is a complex social signal that conveys communicative intent beyond amusement. While prior work has focused on isolated laughter analysis tasks, a comprehensive understanding of laughter in real-world scenarios remains underexplored. Therefore, we introduce SMILE-Next, a dataset for real-world laughter understanding with multimodal textual representations and question-answer annotations across three tasks: laughter detection, laughter type classification, and laughter reasoning. Building upon SMILE-Next, we aim to develop a laughter-specialized large language model capable of nuanced understanding of laughter in real-world contexts. To this end, we propose two key components: laughter-specific Self-Instruct and the Mixture-of-Laugh-Experts (MoLE) framework. Laughter-specific Self-Instruct enhances generalization across tasks and domains by automatically synthesizing diverse laughter-centric instructions. MoLE introduces a task-adaptive expert routing mechanism that dynamically selects specialized experts tailored to each laughter-related task, improving task-specific performance and efficiency. Experimental results show that the combination of our proposed components substantially outperforms multimodal LLM baselines, advancing robust real-world laughter understanding. Project page is at: https://mok0102.github.io/smile-next/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。