arXiv:2505.20278cs.LGcs.AI2025-05被引 4

揭示大模型模式匹配的边界与局限,提出可验证的理论框架。

Characterizing Pattern Matching and Its Limits on Compositional Task Structures

  • 将模式匹配形式化为功能等价性,量化其学习条件。
  • 实证发现:上下文数量决定模式匹配成功率,数据量需20倍参数提升才有效。
  • 多路径依赖导致表征混乱,思维链无法解决此结构性障碍。

尽管大语言模型表现出色,但其成功常依赖于模式匹配行为,而这也导致在组合任务中出现分布外泛化失败。现有研究因任务设计允许多种泛化机制共存,难以精确评估模式匹配的有效性与局限。为此,本文首次将模式匹配形式化为功能等价性——即在输入其余部分固定时,某些子序列始终产生相同输出。通过控制变量的组合任务,系统研究了仅解码器型Transformer与Mamba的表现。理论与实证表明:(1)单个实例的成功率由见证该功能等价性的上下文数量决定;(2)我们证明了学习双跳结构的样本复杂度紧界,且数据缩放律指数与实验结果一致,在20倍参数缩放下跨架构成立;(3)路径歧义是结构性瓶颈:当变量通过多条路径影响输出时,模型无法形成统一中间表示,导致准确率下降、可解释性变差;(4)思维链虽降低数据需求,但不能克服路径歧义。本研究提供了可预测、可证伪的模式匹配边界,为分离混合泛化机制提供基础诊断工具。

原文摘要 · Abstract (English)

Despite impressive capabilities, LLMs' successes often rely on pattern-matching behaviors, yet these are also linked to OOD generalization failures in compositional tasks. However, behavioral studies commonly employ task setups that allow multiple generalization sources (e.g., algebraic invariances, structural repetition), obscuring a precise and testable account of how well LLMs perform generalization through pattern matching and their limitations. To address this ambiguity, we first formalize pattern matching as functional equivalence, i.e., identifying pairs of subsequences of inputs that consistently lead to identical results when the rest of the input is held constant. Then, we systematically study how decoder-only Transformer and Mamba behave in controlled tasks with compositional structures that isolate this mechanism. Our formalism yields predictive and quantitative insights: (1) Instance-wise success of pattern matching is well predicted by the number of contexts witnessing the relevant functional equivalence. (2) We prove a tight sample complexity bound of learning a two-hop structure by identifying the exponent of the data scaling law for perfect in-domain generalization. Our empirical results align with the theoretical prediction, under 20x parameter scaling and across architectures. (3) Path ambiguity is a structural barrier: when a variable influences the output via multiple paths, models fail to form unified intermediate state representations, impairing accuracy and interpretability. (4) Chain-of-Thought reduces data requirements yet does not resolve path ambiguity. Hence, we provide a predictive, falsifiable boundary for pattern matching and a foundational diagnostic for disentangling mixed generalization mechanisms.

大模型机制模式匹配泛化能力可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。