研究对话游戏间知识迁移，发现视觉空间类游戏最易共享能力。
Investigating Knowledge Transfer Across Interactive Dialogue Games
- 用任务转移图和任务向量分析不同对话游戏的知识迁移
- 视觉空间类游戏（如探索类）迁移效果最佳，部分任务靠迁移优于微调
- 基于相似性的方法难捕捉迁移规律，需更复杂评估指标
对话游戏是需要复杂认知能力的挑战性场景，语言既是理解规则的接口，也是执行动作的手段。我们假设在特定语言游戏中训练会提升可迁移到其他任务的能力。为此，本文研究不同对话游戏间的知识迁移。在clembench基准上微调大语言模型，并开展两项分析：一、通过二值整数优化程序构建任务转移图，以任务表现作为核心指标；二、计算每个游戏的任务向量，分析微调后模型间的相似性与迁移能力。结果表明，某些游戏从迁移中获益显著，尤其是视觉空间类（如探索类）游戏迁移效果最好。而任务向量分析显示，基于相似性的方法能捕捉游戏角色关系，但几乎无法反映迁移模式，暗示需要更复杂的评估机制。
原文摘要 · Abstract (English)
Dialogue games represent a challenging setting where complex cognitive skills are required to accomplish tasks while coordinating with other players. Considering that language represents an interface for both understanding the game rules and executing actions, it is reasonable to assume that training on a specific language game will enhance specific capabilities that might be relevant for other tasks as well. Motivated by this rationale, in this paper, we investigate how knowledge transfers across different dialogue games. We study transferability by finetuning LLM models on games from the clembench suite (Chalamalasetti et al., 2023) and performing two analyses: i) we derive a task-transferability graph using a binary integer optimization program from Zamir et al. (2018), using task performance as the main metric; and ii) we compute task vectors (Ilharco et al., 2022) for each game to study similarities across finetuned models and their task transferability. In our first analysis, we find that some games benefit more from transfer than finetuning, and that the visuospatial family (e.g., exploration games) transfers best. With our task vector analysis instead, we find that similarity-based approaches capture game-role relationships but almost no transferability patterns, suggesting that more complex metrics are required.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。