提出显式边机制的图模型，提升逻辑推理能力
Graph Machine: Exploring Edge Mechanisms as an Inductive Bias
- 用边增强注意力和边中心引用，动态构建关系图
- 在数独任务中超越Transformer，提升显著
- 适合需要逻辑推理与关系建模的任务
Transformers虽擅长全局内容匹配，但推理任务可能需更强的迭代关系遍历先验。我们提出Graph Machine，包含两种显式边机制：边增强注意力（边调节节点间注意力）和边中心引用（节点交换地址以更新边）。该设计使模型能跨层动态、可微地构建与修正关系图。在受控的数独任务中评估该先验，结果表明Graph Machine优于Transformer基线；消融实验与机制分析证实性能提升源于边机制。意外发现模型自发构建出紧凑的边结构来表征数独几何。结果支持显式边机制作为有前景的架构设计，值得更广泛评估。
原文摘要 · Abstract (English)
Transformers provide a powerful architecture for global content-based matching, but reasoning problems may benefit from a stronger inductive bias toward iterative traversal of latent relations. We introduce Graph Machine, an architecture with two explicit edge-based mechanisms: Edge-augmented attention, in which edges modulate attention between nodes, and edge-centric referral, in which nodes exchange addresses to update their edges. Conceptually, this enables the model to dynamically and differentiably construct and revise relational graphs across layers. We study this inductive bias using Sudoku under controlled settings and find that Graph Machine outperforms Transformer baselines, with ablation studies and mechanistic analysis attributing the gains to the edge mechanisms. Surprisingly, we found that the model discovers a compact edge-based construction for Sudoku geometry. Our results support explicit edge mechanisms as a promising architectural design, motivating broader evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。