将知识图谱与语言模型结合，实现可解释的联合建模。
Knowledge Graph and Hypergraph Transformers with Repository-Attention and Journey-Based Role Transport
- 用角色槽编码结构数据，构建可查询的键值仓库
- 通过旅程式角色传递统一处理图遍历与句子结构
- 适合需要可解释性与知识对齐的多模态应用
我们提出一种简洁架构，支持文本与结构化数据的联合训练，同时保持语言与知识表示的分离。模型将知识图谱与超图视为带角色槽的结构实例，将其编码为语言变换器可关注的键值仓库。注意力机制基于旅程式角色传递，统一了边标记的知识图遍历、超边遍历与句子结构处理。采用双流架构，包含实例局部、邻域与全局混合注意力的分层层组，独立仓库检索，以及涵盖掩码语言建模、链接预测与角色一致性去噪的多任务目标。结果实现了语言上下文与结构化知识的显式可区分分离，仍可通过交叉注意力实现紧密对齐。
原文摘要 · Abstract (English)
We present a concise architecture for joint training on sentences and structured data while keeping knowledge and language representations separable. The model treats knowledge graphs and hypergraphs as structured instances with role slots and encodes them into a key-value repository that a language transformer can attend over. Attention is conditioned by journey-based role transport, which unifies edge-labeled KG traversal, hyperedge traversal, and sentence structure. We outline a dual-stream architecture, hierarchical layer groups with instance-local, neighborhood, and global mixing attention, retrieval over a separate repository, and multi-task objectives spanning masked language modeling, link prediction, and role-consistency denoising. The result is an explicit, inspectable separation between linguistic context and structured knowledge, while still enabling tight alignment through cross-attention.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。