不需训练即可检测社交媒体情感模型的时间漂移，准确率下降23.4%。
Zero-Training Temporal Drift Detection for Transformer Sentiment Models: A Comprehensive Analysis on Authentic Social Media Streams
- 基于零训练方法，通过新指标实时检测模型漂移
- 事件期间准确率最高下降23.4%，置信度下降13.0%
- 适合需要实时监测的舆情系统部署
我们对基于Transformer的情感模型在真实社交媒体数据上的时间漂移进行了全面的零训练分析，涵盖三大Transformer架构，并在12,279条真实社交帖子上进行严格统计验证。结果显示，在事件驱动时期模型稳定性显著下降，准确率最高降低23.4%。分析发现置信度最大下降13.0%（Bootstrap 95% CI: [9.1%, 16.5%]），与实际性能退化强相关。我们提出四种新型漂移检测指标，优于基于嵌入的基线方法，同时保持生产级部署所需的计算效率。多事件统计验证表明该方法具备鲁棒检测能力，实际意义超过行业监控阈值。该零训练方法可直接用于实时情感监测系统，为动态内容期的Transformer行为提供新洞察。
原文摘要 · Abstract (English)
We present a comprehensive zero-training temporal drift analysis of transformer-based sentiment models validated on authentic social media data from major real-world events. Through systematic evaluation across three transformer architectures and rigorous statistical validation on 12,279 authentic social media posts, we demonstrate significant model instability with accuracy drops reaching 23.4% during event-driven periods. Our analysis reveals maximum confidence drops of 13.0% (Bootstrap 95% CI: [9.1%, 16.5%]) with strong correlation to actual performance degradation. We introduce four novel drift metrics that outperform embedding-based baselines while maintaining computational efficiency suitable for production deployment. Statistical validation across multiple events confirms robust detection capabilities with practical significance exceeding industry monitoring thresholds. This zero-training methodology enables immediate deployment for real-time sentiment monitoring systems and provides new insights into transformer model behavior during dynamic content periods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。