RiverText让词向量能实时更新,适合处理社交媒体等动态文本流。
RiverText: A Python Library for Training and Evaluating Incremental Word Embeddings from Text Data Streams
- 基于流式数据设计,支持动态更新词向量表示
- 集成Skip-gram、CBOW等模型,兼容PyTorch训练框架
- 适配静态评估任务到流式场景,便于对比分析
词向量已成为信息检索与自然语言处理任务中的核心组件,如排序、文档分类和问答系统。然而,传统词向量模型具有静态特性,难以适应社交媒体和网络中不断演变的语言模式(如新话题标签或品牌名)。为此,增量词向量算法被提出,可在持续数据流中动态更新词表示。本文介绍RiverText,一个用于从文本数据流中训练与评估增量词向量的Python库。该工具为信息检索与自然语言处理领域在流式场景下使用词向量提供支持,例如社交媒体分析。库中实现了多种增量词向量技术,包括跳字模型(Skip-gram)、连续词袋(CBOW)和词上下文矩阵,并采用PyTorch作为神经网络训练后端。我们还开发了一个模块,将现有的静态词向量内在评估任务(词相似性与词分类)适配至流式设置。最后,对不同超参数下的方法进行了比较与分析。开源代码已发布于https://github.com/dccuchile/rivertext。
原文摘要 · Abstract (English)
Word embeddings have become essential components in various information retrieval and natural language processing tasks, such as ranking, document classification, and question answering. However, despite their widespread use, traditional word embedding models present a limitation in their static nature, which hampers their ability to adapt to the constantly evolving language patterns that emerge in sources such as social media and the web (e.g., new hashtags or brand names). To overcome this problem, incremental word embedding algorithms are introduced, capable of dynamically updating word representations in response to new language patterns and processing continuous data streams. This paper presents RiverText, a Python library for training and evaluating incremental word embeddings from text data streams. Our tool is a resource for the information retrieval and natural language processing communities that work with word embeddings in streaming scenarios, such as analyzing social media. The library implements different incremental word embedding techniques, such as Skip-gram, Continuous Bag of Words, and Word Context Matrix, in a standardized framework. In addition, it uses PyTorch as its backend for neural network training. We have implemented a module that adapts existing intrinsic static word embedding evaluation tasks for word similarity and word categorization to a streaming setting. Finally, we compare the implemented methods with different hyperparameter settings and discuss the results. Our open-source library is available at https://github.com/dccuchile/rivertext.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。