arXiv:2411.12484cs.LGcs.CL2024-11

让条件随机场学会捕捉远距离标签关系,用户只需写正则模式即可。

Regular-pattern-sensitive CRFs for Distant Label Interactions

  • 用用户指定的正则模式扩展传统CRF,引入长程标签交互能力。
  • 在三个合成数据集上验证有效,能准确学习并利用指定模式中的交互。
  • 模型结构可解释,训练与推断对多数模式组合仍高效可行。

尽管大语言模型在序列标注中日益流行,线性链条件随机场(CRFs)仍因其能直接建模标签间相互作用而备受青睐。然而,马尔可夫假设限制其仅能建模相邻标签间的交互。加权有限状态转换器(FSTs)虽可建模远距离标签交互,但一般情况下精确推理不可行。本文提出正则模式敏感的条件随机场(RPCRFs),通过用户指定的模式,使标准线性链CRF具备学习长距离标签交互的能力。用户可用简洁的正则表达式指定希望模型关注的标签交互类型,模型则从数据中学习这些模式是否出现及在何种上下文中出现。该方法可视为带有额外非局部势能的CRF,或由易解释模式定义结构的有限状态转换器。关键的是,对于许多模式集合,精确训练和推理仍为可处理。我们详述了如何从一组用户指定模式自动生成RPCRF,并在三个合成序列建模数据集上展示了模型的有效性。

原文摘要 · Abstract (English)

While LLMs have grown popular in sequence labeling, linear-chain conditional random fields (CRFs) remain a popular alternative with the ability to directly model interactions between labels. However, the Markov assumption limits them to % only directly modeling interactions between adjacent labels. Weighted finite-state transducers (FSTs), in contrast, can model distant label--label interactions, but exact label inference is intractable in general. In this work, we present regular-pattern-sensitive CRFs (RPCRFs), a method of enriching standard linear-chain CRFs with the ability to learn long-distance label interactions through user-specified patterns. This approach allows users to write regular-expression label patterns concisely specifying which types of interactions the model should take into account, allowing the model to learn from data whether and in which contexts these patterns occur. The result can be interpreted alternatively as a CRF augmented with additional, non-local potentials, or as a finite-state transducer whose structure is defined by a set of easily-interpretable patterns. Critically, exact training and inference are tractable for many pattern sets. We detail how an RPCRF can be automatically constructed from a set of user-specified patterns, and demonstrate the model's effectiveness on a sequence of three synthetic sequence modeling datasets.

序列标注条件随机场长程依赖正则模式

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。