用聚类预训练提升少标签场景下的多模态视频情感分类效果
Enhancing Multi-Modal Video Sentiment Classification Through Semi-Supervised Clustering
- 通过聚类实现半监督预训练，从无标签数据中挖掘视频、文本与音频的潜在结构
- 在少量标注数据下仍能达到较高分类准确率，有效缓解标注成本问题
- 适合资源受限但需高精度情感分析的场景，如社交媒体内容理解
视频情感理解是一项具有挑战性的任务，但视频包含多种模态信息，为机器学习与深度学习提供了丰富数据。本文旨在通过关注视频本身、伴随文本及声学特征，提升视频情感分类性能。针对依赖大规模标注数据的局限性，提出基于聚类的半监督预训练方法，以从数据中提取有意义的表征。该预训练阶段能识别视频与文本数据中的模式，使模型在无需大量标注信息的前提下学习到底层结构与关系。随后，在此基础上进行有监督微调，完成情感分类。我们相信，这种结合聚类与有监督微调的多模态方法，能在标注数据有限的情况下实现更准确、更深入的情感分类。
原文摘要 · Abstract (English)
Understanding emotions in videos is a challenging task. However, videos contain several modalities which make them a rich source of data for machine learning and deep learning tasks. In this work, we aim to improve video sentiment classification by focusing on two key aspects: the video itself, the accompanying text, and the acoustic features. To address the limitations of relying on large labeled datasets, we are developing a method that utilizes clustering-based semi-supervised pre-training to extract meaningful representations from the data. This pre-training step identifies patterns in the video and text data, allowing the model to learn underlying structures and relationships without requiring extensive labeled information at the outset. Once these patterns are established, we fine-tune the system in a supervised manner to classify the sentiment expressed in videos. We believe that this multi-modal approach, combining clustering with supervised fine-tuning, will lead to more accurate and insightful sentiment classification, especially in cases where labeled data is limited.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。