arXiv:2412.13745cs.CL2024-12被引 2

用复数向量训练词嵌入,支持经典与量子两种方式。

Learning Complex Word Embeddings in Classical and Quantum Spaces

  • 将经典Skip-gram模型扩展为复数向量,直接替换实数向量。
  • 在400k词汇量上训练38亿词的复数嵌入,性能媲美经典模型。
  • 两阶段量子方法可高效生成高质量量子启发嵌入,适合探索量子NLP优势。

本文提出多种训练复数词嵌入的方法,基于经典Skip-gram模型,通过简单地将实数向量替换为任意复数向量实现。在更“物理启发”的方法中,使用参数化量子电路(PQC)生成向量,其为幺正变换,结果为归一化向量并具有概率解释。我们开发了高度优化的复数版C语言Skip-gram代码,可在包含38亿词的语料库上训练超过40万词汇量的嵌入,并为每个词单独训练一个PQC。在标准相似性与相关性数据集上评估复数嵌入,部分模型表现与经典基线相当。尽管直接训练PQC会降低性能,但两阶段流程生成的量子嵌入在参数数量相近时表现媲美经典模型。这提供了一种可扩展的复杂空间嵌入学习路径,其扩展性取决于词汇量而非语料规模。综上,本工作展示了大规模高质量复数与量子启发词嵌入的生成方法,适用于复杂值及量子启发的NLP模型,并探索量子NLP潜在优势。

原文摘要 · Abstract (English)

We present a variety of methods for training complex-valued word embeddings, based on the classical Skip-gram model, with a straightforward adaptation simply replacing the real-valued vectors with arbitrary vectors of complex numbers. In a more "physically-inspired" approach, the vectors are produced by parameterised quantum circuits (PQCs), which are unitary transformations resulting in normalised vectors which have a probabilistic interpretation. We develop a complex-valued version of the highly optimised C code version of Skip-gram, which allows us to easily produce complex embeddings trained on a 3.8B-word corpus for a vocabulary size of over 400k, for which we are then able to train a separate PQC for each word. We evaluate the complex embeddings on a set of standard similarity and relatedness datasets, for some models obtaining results competitive with the classical baseline. We find that, while training the PQCs directly tends to harm performance, the quantum word embeddings from the two-stage process perform as well as the classical Skip-gram embeddings with comparable numbers of parameters. This enables a highly scalable route to learning embeddings in complex spaces which scales with the size of the vocabulary rather than the size of the training corpus. In summary, we demonstrate how to produce a large set of high-quality word embeddings for use in complex-valued and quantum-inspired NLP models, and for exploring potential advantage in quantum NLP models.

词嵌入复数向量量子计算NLP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。