arXiv:2606.12018cs.AI2026-06被引 1

用轻量多模态大模型+知识蒸馏,让AI更懂社交中的长尾细节。

MODF-SIR: A Multi-agent Omni-modal Distilled Framework for Social Intelligence Reasoning

论文配图:MODF-SIR: A Multi-agent Omni-modal Distilled Framework for Social Intelligence Reasoning
图 1 · 摘自论文原文
  • 多智能体协作框架,结合知识蒸馏与测试时自适应提升推理能力。
  • 在多个基准上仅用30%训练数据即达顶尖性能,长尾事件识别准确率高。
  • 适合需要精准社交理解的场景,如智能客服、虚拟助手等应用者。

我们提出一种基于轻量级多模态大语言模型(MLLM)的多智能体协同框架,专为社会智能推理设计。该方法在训练和推理阶段均通过知识蒸馏增强。框架能精确定位与社会智能相关的多模态数据,并识别、提取长尾事件,将其格式化为显式文本,避免其在分词过程中被主流事件和环境噪声掩盖。整个推理流程集成测试时自适应(TTA),涵盖长尾事件的提取与表征、思维链(CoT)提示及自我反思环节,且该机制亦经蒸馏优化,采用低秩适配(LoRA)对基础模型进行实例级微调。在多个公开与私有模型的对比评估中表现优异,仅使用约30%的IntentTrain数据即达到当前最优结果。代码、演示、LoRA权重及路由训练数据集均已开源。

原文摘要 · Abstract (English)

We propose a multi-agent collaborative framework built upon a lightweight Multimodal Large Language Model (MLLM), specifically designed for social intelligence reasoning. A key feature of our approach is that both the training and inference phases are augmented via knowledge distillation. Within this architecture, multi-modal data pertinent to social intelligence is precisely localized. Furthermore, relevant long-tail events are identified, extracted, and rendered as formatted, explicit text. This formatting strategy prevents critical long-tail information from being overshadowed by head events and environmental noise during the tokenization process. Specifically, we integrate Test-Time Adaptation (TTA) across the entire reasoning pipeline, encompassing the extraction and representation of long-tail events, Chain-of-Thought (CoT) prompting, and self-reflection. This TTA mechanism is also distillation-enhanced, utilizing Low-Rank Adaptation (LoRA) to fine-tune the foundation model exclusively for instance-level reasoning. Extensive evaluations against various open-source and proprietary AI models across multiple benchmarks demonstrate the effectiveness of the proposed framework. With around 30% of training data from IntentTrain, we achieve state-of-the-art results. Codes are available at https://github.com/eeee-sys/MODF-SIR, demo is available at https://huggingface.co/spaces/Harry-1234/MODF-SIR, LoRA is available at https://huggingface.co/Harry-1234/MODF-SIR and the dataset for training router is available at https://huggingface.co/datasets/Harry-1234/IntentRouterTrain.

多模态社会推理知识蒸馏长尾事件

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。