MLego通过复用模型实现实时主题探索,提升分析效率。
MLego: Interactive and Scalable Topic Exploration Through Model Reuse
- 复用已训练模型,避免重复计算,加速主题建模。
- 交互式查询速度提升显著,计算成本降低超过50%。
- 适合需要快速洞察海量文本的分析师与数据可视化用户。
社交媒体上的海量文本使得用户和分析师常依赖主题建模技术快速提取关键主题并获得洞见。传统方法如隐含狄利克雷分布(LDA)虽能提供有价值的信息,但计算开销大,难以支持实时分析。尽管分布式训练和快速采样方法有所改进,实时主题探索仍是重大挑战。本文提出MLego,一个通过模型材料化与复用支持实时主题建模分析的交互式查询框架。无需从头训练,MLego通过合并已材料化的主题模型,以近似方式在交互速度下生成结果。为进一步提升效率,引入分层查询计划搜索策略用于单个查询,以及优化的批处理查询重排序技术。我们将MLego集成至视觉分析原型系统,使用户可通过交互查询探索大规模文本数据集。大量实验表明,MLego显著降低计算成本,同时保持高质量的主题建模结果。它拓展了现有视觉分析方法,不仅支持用户驱动的主题建模,更实现真正意义上的实时、查询驱动探索,弥合了可扩展主题建模与交互式数据分析之间的差距。
原文摘要 · Abstract (English)
With massive texts on social media, users and analysts often rely on topic modeling techniques to quickly extract key themes and gain insights. Traditional topic modeling techniques, such as Latent Dirichlet Allocation (LDA), provide valuable insights but are computationally expensive, making them impractical for real-time data analysis. Although recent advances in distributed training and fast sampling methods have improved efficiency, real-time topic exploration remains a significant challenge. In this paper, we present MLego, an interactive query framework designed to support real-time topic modeling analysis by leveraging model materialization and reuse. Instead of retraining models from scratch, MLego efficiently merges materialized topic models to construct approximate results at interactive speeds. To further enhance efficiency, we introduce a hierarchical plan search strategy for single queries and an optimized query reordering technique for batch queries. We integrate MLego into a visual analytics prototype system, enabling users to explore large-scale textual datasets through interactive queries. Extensive experiments demonstrate that MLego significantly reduces computation costs while maintaining high-quality topic modeling results. MLego enhances existing visual analytics approaches, which primarily focus on user-driven topic modeling, by enabling real-time, query-driven exploration. This complements traditional methods and bridges the gap between scalable topic modeling and interactive data analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。