用高斯过程构建连续潜空间，实现语言生成的并行化与时序结构保持。
Latent-Autoregressive GP-VAE Language Model
- 将语言时序动态转移至连续潜空间，用高斯过程建模序列依赖。
- 在受限框架下训练稳定，自回归与并行采样表现一致。
- 适合对潜空间几何建模感兴趣的生成模型研究者。
我们提出一种完全基于潜空间的自回归方案，将高斯过程（GP）融入变分自编码器（VAE）。在此框架中，序列动态从观测空间转移到连续潜空间，而语言生成仍通过非自回归解码器并行完成。本文给出了完整的方法学设计，包括因果高斯过程先验、结构化压缩后验以及基于正则化证据下界（ELBO）的训练协议。在刻意限制的验证性实验框架中，模型表现出稳定的训练能力，自回归与并行采样版本的行为具有一致性。结果表明，语言模型中的部分时序结构可由潜空间的概率几何支撑，而非依赖显式的神经运算。
原文摘要 · Abstract (English)
We investigate a fully Latent AutoRegressive scheme based on a Gaussian Process (GP) integrated into a Variational Autoencoder (VAE). In this setting, sequential dynamics are transferred from the observation space to a continuous latent space, while linguistic generation remains parallel through a non-autoregressive decoder. We present a complete methodological formulation, including a causal GP prior, a structured amortized posterior, and a training protocol based on a regularized ELBO. Empirical evaluation, conducted within a deliberately constrained proof-of-concept (POC) framework, shows that the model can be trained stably and that the sequential and parallel sampling variants exhibit consistent behavior. Overall, the results suggest that part of the temporal structure in a language model can be supported by the probabilistic geometry of the latent space rather than by explicit neural operations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。