arXiv:2501.01422cs.CVcs.AI2025-01被引 1

用多模态特征预测短视频受欢迎程度,效果优于单一模型。

Multi-Modal Video Feature Extraction for Popularity Prediction

  • 融合视频分类与文本生成,提取六类特征向量。
  • 结合神经网络与XGBoost,平均预测提升稳定性。
  • 适合关注短视频推荐与内容分析的研究者。

本文旨在通过视频本身及其相关特征预测短视频的受欢迎程度,以观看量、点赞数、评论数和分享数四个关键互动指标衡量热度。研究采用不同架构与训练方式的视频分类模型作为主干网络,提取视频模态特征;同时,将清洗后的视频字幕融入精心设计的提示框架,与视频一同输入视频-文本生成模型,生成详细的文本化视频理解内容,并使用预训练BERT模型将其编码为向量。基于上述六组向量,分别针对四个预测指标训练神经网络。此外,结合视频与表格数据进行数据挖掘与特征工程,构建了标签出现总频次、提及出现总频次、视频时长、帧数、帧率及在线总时长等实用特征。训练多种机器学习模型后,选取表现最稳定的XGBoost模型。最终,将神经网络与XGBoost的预测结果取平均,得到最终预测结果。

原文摘要 · Abstract (English)

This work aims to predict the popularity of short videos using the videos themselves and their related features. Popularity is measured by four key engagement metrics: view count, like count, comment count, and share count. This study employs video classification models with different architectures and training methods as backbone networks to extract video modality features. Meanwhile, the cleaned video captions are incorporated into a carefully designed prompt framework, along with the video, as input for video-to-text generation models, which generate detailed text-based video content understanding. These texts are then encoded into vectors using a pre-trained BERT model. Based on the six sets of vectors mentioned above, a neural network is trained for each of the four prediction metrics. Moreover, the study conducts data mining and feature engineering based on the video and tabular data, constructing practical features such as the total frequency of hashtag appearances, the total frequency of mention appearances, video duration, frame count, frame rate, and total time online. Multiple machine learning models are trained, and the most stable model, XGBoost, is selected. Finally, the predictions from the neural network and XGBoost models are averaged to obtain the final result.

视频预测多模态流行度分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。