用在线密度聚类提升社交媒体叙事监控的实时性与可扩展性
Online Density-Based Clustering for Real-Time Narrative Evolution Monitorin
- 采用在线密度聚类替代传统批处理方法，实现流式数据持续处理
- 在乌克兰信息空间数据上验证，DenStream在稳定性和叙事连贯性间表现最优
- 兼顾聚类质量与下游叙事生成需求，适合大规模实时舆情系统
依赖批量聚类方法的社会媒体自动叙事智能系统在处理连续数据流时面临显著可扩展性挑战。本文研究在生产级叙事报告生成流水线中，以在线密度聚类算法替代离线的HDBSCAN，处理大规模多语言社交数据。尽管HDBSCAN能有效发现层次化聚类并处理噪声，但其仅支持批量处理，每个时间窗口均需完整重训练，限制了系统的可扩展性与实时适应能力。我们评估了多种在线聚类方法在聚类质量、计算效率、内存开销及与下游叙事提取集成方面的表现。评估结合标准聚类指标、叙事特异性度量及人工验证，综合评估结构质量与语义可解释性。基于乌克兰信息空间历史数据的滑动窗口模拟实验揭示了时间稳定性与叙事连贯性间的权衡，其中DenStream表现出最强整体性能。研究弥合了批处理聚类方法与大规模叙事监控系统流式需求之间的差距。
原文摘要 · Abstract (English)
Automated narrative intelligence systems for social media monitoring face significant scalability challenges when relying on batch clustering methods to process continuous data streams. We investigate replacing offline HDBSCAN with online density-based clustering algorithms in a production narrative report generation pipeline that processes large volumes of multilingual social media data. While HDBSCAN effectively discovers hierarchical clusters and handles noise, its batch-only nature requires full retraining for each time window, limiting scalability and real-time adaptability. We evaluate online clustering methods with respect to cluster quality, computational efficiency, memory footprint, and integration with downstream narrative extraction. Our evaluation combines standard clustering metrics, narrative-specific measures, and human validation of cluster correctness to assess both structural quality and semantic interpretability. Experiments using sliding-window simulations on historical data from the Ukrainian information space reveal trade-offs between temporal stability and narrative coherence, with DenStream achieving the strongest overall performance. These findings bridge the gap between batch-oriented clustering approaches and the streaming requirements of large-scale narrative monitoring systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。