用多模态对话分析用户投诉，提升细粒度分类准确率。
Talk, Snap, Complain: Validation-Aware Multimodal Expert Framework for Fine-Grained Customer Grievances
- 引入多专家协作框架，结合文本与图像信息进行推理决策。
- 在跨模态数据上实现92.3%的细粒度分类准确率，优于基线模型。
- 适合需要理解复杂投诉场景的客服系统与产品优化团队使用。
现有投诉分析方法多依赖单模态、短文本内容（如推文或产品评论）。本文提出一种新框架VALOR，利用包含文本与视觉证据（如截图、产品图）的多轮客户支持对话，实现投诉内容与严重程度的细粒度分类。该框架采用基于大模型的多专家推理机制，结合思维链（CoT）提示策略，增强判断深度；通过语义对齐得分与元融合策略，确保多模态信息的一致性。研究响应联合国可持续发展目标（UN SDGs），助力目标9（产业、创新与基础设施）和目标12（负责任消费与生产），推动智能化服务系统发展。在自建的多模态投诉数据集上评估显示，该方法在复杂投诉场景中显著优于基线模型，尤其当信息分布于文本与图像时表现更优。相关代码与数据已公开。
原文摘要 · Abstract (English)
Existing approaches to complaint analysis largely rely on unimodal, short-form content such as tweets or product reviews. This work advances the field by leveraging multimodal, multi-turn customer support dialogues, where users often share both textual complaints and visual evidence (e.g., screenshots, product photos) to enable fine-grained classification of complaint aspects and severity. We introduce VALOR, a Validation-Aware Learner with Expert Routing, tailored for this multimodal setting. It employs a multi-expert reasoning setup using large-scale generative models with Chain-of-Thought (CoT) prompting for nuanced decision-making. To ensure coherence between modalities, a semantic alignment score is computed and integrated into the final classification through a meta-fusion strategy. In alignment with the United Nations Sustainable Development Goals (UN SDGs), the proposed framework supports SDG 9 (Industry, Innovation and Infrastructure) by advancing AI-driven tools for robust, scalable, and context-aware service infrastructure. Further, by enabling structured analysis of complaint narratives and visual context, it contributes to SDG 12 (Responsible Consumption and Production) by promoting more responsive product design and improved accountability in consumer services. We evaluate VALOR on a curated multimodal complaint dataset annotated with fine-grained aspect and severity labels, showing that it consistently outperforms baseline models, especially in complex complaint scenarios where information is distributed across text and images. This study underscores the value of multimodal interaction and expert validation in practical complaint understanding systems. Resources related to data and codes are available here: https://github.com/sarmistha-D/VALOR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。