arXiv:2607.23333cs.LGcs.AI2026-07

用后悔损失训练单层自注意力模型,实现博弈论中的均衡策略。

Training with (Swap) Regret Loss in a Single-Layer Self-Attention Model: A Case Study on the Probability Simplex

论文配图:Training with (Swap) Regret Loss in a Single-Layer Self-Attention Model: A Case Study on the Probability Simplex
图 1 · 摘自论文原文
  • 用后悔损失训练模型,使其输出等同于平滑虚构博弈的更新。
  • 引入交换后悔损失,使模型能抵御策略偏差,逼近相关均衡。
  • 无需监督信号,小模型即可实现在线学习的博弈均衡行为。

我们重新审视 Park 等人(2025)提出的后悔损失框架,将其应用于概率单纯形上的策略建模。结果表明,使用后悔损失训练的单层自注意力模型存在一个驻点,其前向传播恰好对应于步长合适的平滑虚构博弈更新——对任意策略输入,模型输出与平滑虚构博弈的更新一致。同时,我们新提出一种交换后悔损失函数,将后悔损失框架扩展至交换后悔,使模型可直接优化对交换偏离的鲁棒性。进一步证明,该交换后悔损失也存在驻点,其前向传播实现了经典 Blum-Mansour 无回溯算法所诱导的交换后悔更新,每个注意力头通过平滑虚构博弈实现外部后悔更新。这些结果表明,基于后悔的目标可引导极简注意力架构生成可微机制,部署后在博弈中产生均衡行为:外部后悔动态收敛至粗略相关均衡,交换后悔动态收敛至相关均衡。因此,后悔目标在无需监督信号的情况下,将小型注意力模型导向具有博弈论保证的在线学习动态。

原文摘要 · Abstract (English)

We revisit the regret loss framework introduced in Park et al. (2025), which uses decision-theoretic regret as a direct loss function for training models to make better decisions, through the lens of probability-simplex policies. Our first result shows that a single-layer self-attention model trained with regret loss admits a stationary point whose forward-pass exactly matches smoothed fictitious play with the appropriate stepsize that ensures no-regret behavior-i.e., for any given policy input, the model outputs the same update that smoothed fictitious play would produce. In parallel, we also newly introduce a swap-regret loss function, which extends the regret-loss framework beyond external regret and enables models to directly optimize for swap-deviation robustness. We further show that this swap-regret loss admits a stationary point whose forward pass implements the corresponding swap-regret update induced by classical Blum-Mansour no-pass implementation algorithm, with each head implementing an external-regret update via smoothed fictitious play. Together, these results show that regret-trained attention can realize differentiable mechanisms whose deployment induces equilibrium behavior in games: external-regret dynamics lead to coarse correlated equilibrium, while swap-regret dynamics lead to correlated equilibrium. Thus, regret-based objectives steer minimal attention architectures toward online-learning dynamics with game-theoretic guarantees, without supervised traces of those algorithms.

自注意力后悔损失博弈论均衡学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。