用用户评论和网络数据融合评估视频体验质量,精准发现服务异常。
Bridging Subjective and Objective QoE: Operator-Level Aggregation Using LLM-Based Comment Analysis and Network MOS Comparison
- 结合网络参数与直播评论,用大模型提取主观体验评分。
- 构建4.7万条标注评论数据集,实现运营商级质量趋势分析。
- 可实时比对主观与客观质量,快速定位局部服务问题。
本文提出一种双层网络运营商侧体验质量(QoE)评估框架,融合客观网络建模与主观用户感知。在客观层面,基于ITU-T P.1203标准计算的平均意见分(MOS)训练机器学习模型,仅通过丢包率、延迟、抖动和吞吐量等网络参数即可准确预测用户感知视频质量,无需视频内容或客户端设备数据。在主观层面,设计语义过滤与评分流程,利用大语言模型对直播平台用户评论进行处理,以确定性方式为筛选后的评论分配标量MOS值。构建包含47,894条直播评论的标注数据集,其中约34,000条经多层语义过滤被识别为与QoE相关。每条评论均添加模拟的ISP归属信息,并按5分钟间隔同步合成时间戳。该数据集支持运营商级别的聚合与时间序列分析。提出增量MOS指标,用于衡量各运营商相对于平台整体情绪的偏离程度,可在缺乏直接网络遥测的情况下检测局部退化。受控断网仿真验证了该框架仅通过评论趋势即可有效识别服务中断。系统为每个运营商提供其自身的主观MOS及每时段平台均值,实现性能偏差的实时解读,并与基于网络的客观QoE估计进行对比。
原文摘要 · Abstract (English)
This paper introduces a dual-layer framework for network operator-side quality of experience (QoE) assessment that integrates both objective network modeling and subjective user perception extracted from live-streaming platforms. On the objective side, we develop a machine learning model trained on mean opinion scores (MOS) computed via the ITU-T P.1203 reference implementation, allowing accurate prediction of user-perceived video quality using only network parameters such as packet loss, delay, jitter, and throughput without reliance on video content or client-side instrumentation. On the subjective side, we present a semantic filtering and scoring pipeline that processes user comments from live streams to extract performance-related feedback. A large language model is used to assign scalar MOS scores to filtered comments in a deterministic and reproducible manner. To support scalable and interpretable analysis, we construct a labeled dataset of 47,894 live-stream comments, of which about 34,000 are identified as QoE-relevant through multi-layer semantic filtering. Each comment is enriched with simulated Internet Service Provider attribution and temporally aligned using synthetic timestamps in 5-min intervals. The resulting dataset enables operator-level aggregation and time-series analysis of user-perceived quality. A delta MOS metric is proposed to measure each Internet service provider's deviation from platform-wide sentiment, allowing detection of localized degradations even in the absence of direct network telemetry. A controlled outage simulation confirms the framework's effectiveness in identifying service disruptions through comment-based trends alone. The system provides each operator with its own subjective MOS and the global platform average per interval, enabling real-time interpretation of performance deviations and comparison with objective network-based QoE estimates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。