XLM开源工具包加速非自回归语言模型开发，支持快速实验与复用。
XLM: A Python package for non-autoregressive language models
- 提供统一框架，简化非自回归模型的数据处理与损失计算
- 支持小规模模型快速实现，降低实验门槛
- 配套预训练模型，适合研究者快速验证新方法
近年来，非自回归文本生成在通用语言建模中重获关注。与成熟的自回归建模范式不同，非自回归语言建模的实现多为定制化，缺乏标准训练与推理库，导致方法间难以系统比较。此外，每种模型通常需独立设计数据整理、损失函数和预测逻辑，组件复用困难。本文提出XLM Python工具包，旨在加速小型非自回归语言模型的实现，次要目标是通过配套的xlm-models包提供一系列小型预训练模型，供研究社区使用。代码已公开于https://github.com/dhruvdcoder/xlm-core。
原文摘要 · Abstract (English)
In recent years, there has been a resurgence of interest in non-autoregressive text generation in the context of general language modeling. Unlike the well-established autoregressive language modeling paradigm, which has a plethora of standard training and inference libraries, implementations of non-autoregressive language modeling have largely been bespoke making it difficult to perform systematic comparisons of different methods. Moreover, each non-autoregressive language model typically requires it own data collation, loss, and prediction logic, making it challenging to reuse common components. In this work, we present the XLM python package, which is designed to make implementing small non-autoregressive language models faster with a secondary goal of providing a suite of small pre-trained models (through a companion xlm-models package) that can be used by the research community. The code is available at https://github.com/dhruvdcoder/xlm-core.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。