提出精确生成带约束的变阶马尔可夫序列的新方法
Exact Regular-Constrained Variable-Order Markov Generation via Sparse Context-State Belief Propagation
- 用上下文状态替换一阶状态,构建稀疏产品结构
- 在约束条件下保持变阶模型正确分布,推理时间线性
- 适合需要精确生成与可逆数据增强的序列建模任务
变阶马尔可夫模型通过依赖已生成历史的最长后缀来生成有限字母表上的序列。而正则约束则通过自动机描述有限时域控制要求,如固定位置、强制结尾、节拍模式和禁止重复片段等。现有精确方法已能对一阶马尔可夫链施加正则约束,本文将其扩展至变阶模型:识别出必须运行现有BP-正则机制的状态空间。一阶约束层虽能施加有效支持条件,但会合并变阶生成器有意区分的历史。本文揭示此不匹配并提出解决方案:以观察到的上下文状态替代一阶马尔可夫状态,再与正则约束自动机进行标准乘积构造。对于固定训练的上下文图与自动机,推理复杂度在序列长度上为线性;一般情况下为可达乘积边数的多项式。该方法在不展开所有K元组的前提下,实现条件于正则约束的正确变阶分布。相同有限源接口还支持通过逆向计数查找实现可逆数据增强,无需存储转换后的语料库。同时将精确的BP推理与生成时的回退策略(如单例避免)分离,若声称精确性,则需明确定义其随机语义。
原文摘要 · Abstract (English)
Variable-order Markov models generate sequences over a finite alphabet by conditioning each symbol on the longest available suffix of the generated history. Regular constraints, by contrast, describe finite-horizon control requirements by an automaton: fixed positions, forced endings, metrical patterns, and forbidden copied fragments are all special cases. Existing exact methods already handle regular constraints with belief propagation for first-order Markov chains. The contribution here is the variable-order extension: identifying the state space on which the existing BP-regular machinery must be run when the generator is a variable-order/backoff model. A first-order constraint layer can enforce useful support conditions, but it computes future mass after merging histories that a variable-order generator deliberately keeps distinct. We formalize this mismatch and give the sparse construction obtained by replacing the first-order Markov state with the observed context state, then taking the standard product with the regular constraint automaton. For a fixed trained context graph and automaton, inference is linear in the sequence horizon; in general it is polynomial in the number of reachable product edges. This gives the correct variable-order distribution conditioned on regular constraints without expanding to all K-tuples. The same finite-source interface supports reversible data augmentation by inverse count lookup, matching materialized transposition augmentation without storing transformed corpora. We also separate exact BP inference from generation-time backoff policies, such as singleton avoidance, whose stochastic semantics must be made explicit if exactness is claimed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。