提出两种分解微分博弈的新方法，揭示学习动态与策略空间的深层联系。
On the Decomposition of Differential Game
- 基于赫尔姆霍兹定理，将博弈分解为标量势、向量势和非策略部分
- 标量势博弈下梯度下降可收敛至纳什均衡，向量势博弈则可能发散或循环
- 为理解连续策略博弈的学习行为提供新分析框架，适合博弈论与强化学习研究者
为理解微分博弈中学习动态的复杂性，本文将博弈分解为动态已知的组成部分。利用赫尔姆霍兹定理可将向量场分解为标量势与调和分量，该方法在有限及标准型博弈中已被证明有效。然而，由于微分博弈的策略空间 $\mathbb{R}^n$ 非紧致，直接将其与 $\mathbb{R}^n$ 上的霍奇定理关联存在困难，此前该问题尚未解决 \\cite{letcher2019differentiable}。本文给出两种分解方式：第一种为精确标量势部分、近似向量势部分与非策略部分；第二种为近似标量势部分、精确向量势部分与非策略部分。我们证明标量势博弈等同于蒙德雷与沙尔（Monderer & Shapley, 1996）提出的势博弈，其梯度下降可成功找到纳什均衡。对于向量势博弈，个体梯度场为无散场，此时梯度下降动态可能发散或呈现周期性。
原文摘要 · Abstract (English)
To understand the complexity of the dynamic of learning in differential games, we decompose the game into components where the dynamic is well understood. One of the possible tools is Helmholtz's theorem, which can decompose a vector field into a potential and a harmonic component. This has been shown to be effective in finite and normal-form games. However, applying Helmholtz's theorem by connecting it with the Hodge theorem on $\mathbb{R}^n$ (which is the strategy space of differential game) is non-trivial due to the non-compactness of $\mathbb{R}^n$. Bridging the dynamic-strategic disconnect through Hodge/Helmoltz's theorem in differential games is then left as an open problem \cite{letcher2019differentiable}. In this work, we provide two decompositions of differential games to answer this question: the first as an exact scalar potential part, a near vector potential part, and a non-strategic part; the second as a near scalar potential part, an exact vector potential part, and a non-strategic part. We show that scalar potential games coincide with potential games proposed by \cite{monderer1996potential}, where the gradient descent dynamic can successfully find the Nash equilibrium. For the vector potential game, we show that the individual gradient field is divergence-free, in which case the gradient descent dynamic may either be divergent or recurrent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。