构建金融视频投诉数据集,实现多模态投诉细粒度识别
Deciphering the complaint aspects: Towards an aspect-based complaint identification model with video complaint dataset in finance
- 基于CLIP双冻结编码器+图像分割注意力,融合音视频特征
- 在433条标注视频上实现多标签投诉分类,性能优于现有基线
- 适合金融客服、智能质检等需要理解用户具体投诉场景的领域
在竞争激烈的营销环境中,有效的投诉管理对客户服务和业务成功至关重要。视频投诉结合文本与图像内容,能深入揭示客户诉求,并明确产品优劣。然而,从海量日常多模态金融数据中理解细微的投诉方面仍具挑战。为此,我们构建了一个自有多模态视频投诉数据集,包含433个公开可访问实例,每条数据在话语层级进行标注,涵盖五类金融相关方面及其对应的投诉标签。为支持该任务,我们提出Solution 3.0模型,专用于多模态方面级投诉识别。该模型具备三项核心能力:1)处理音视频多模态特征;2)支持多标签方面分类;3)并行执行方面分类与投诉识别。Solution 3.0采用基于CLIP的双冻结编码器,集成图像分割编码器实现全局特征融合,并引入上下文注意力机制(ISEC)以提升准确率与效率。实验表明,该框架在几乎所有指标上均超越现有多模态基线,为精准客户关怀举措提供新路径,有效辅助用户解决问题。
原文摘要 · Abstract (English)
In today's competitive marketing landscape, effective complaint management is crucial for customer service and business success. Video complaints, integrating text and image content, offer invaluable insights by addressing customer grievances and delineating product benefits and drawbacks. However, comprehending nuanced complaint aspects within vast daily multimodal financial data remains a formidable challenge. Addressing this gap, we have curated a proprietary multimodal video complaint dataset comprising 433 publicly accessible instances. Each instance is meticulously annotated at the utterance level, encompassing five distinct categories of financial aspects and their associated complaint labels. To support this endeavour, we introduce Solution 3.0, a model designed for multimodal aspect-based complaint identification task. Solution 3.0 is tailored to perform three key tasks: 1) handling multimodal features ( audio and video), 2) facilitating multilabel aspect classification, and 3) conducting multitasking for aspect classifications and complaint identification parallelly. Solution 3.0 utilizes a CLIP-based dual frozen encoder with an integrated image segment encoder for global feature fusion, enhanced by contextual attention (ISEC) to improve accuracy and efficiency. Our proposed framework surpasses current multimodal baselines, exhibiting superior performance across nearly all metrics by opening new ways to strengthen appropriate customer care initiatives and effectively assisting individuals in resolving their problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。