将时间作为独立模态,提升多模态知识图谱的实体区分能力。
Time Imprint: Learning Time-Aware Representations in Multi-Modal Knowledge Graphs

- 把时间当作与文本、图像并列的模态,用三模态对比学习对齐表示。
- 在时间戳冗余时选择关键时间点,注意力聚合提升表征鲁棒性与特异性。
- 在高混淆样本上性能提升达58%,适合需要精细时间区分的任务。
多模态知识图谱(MMKG)为实体融合文本和图像等多模态信息,但特征高度相似的实体仍难以区分。时间信息可作为额外模态帮助消歧,但现有方法因两大挑战少将时间视为独立模态:(1)时间语义稀疏,难与丰富模态对齐;(2)多个时间戳引入噪声或降低表示学习鲁棒性。为此,我们提出Time Imprint框架,将时间作为实体级模态,通过三视图对比目标联合对齐时间、文本与视觉表示。为缓解多时间戳歧义,该框架探索紧凑时间戳子集选择的设计空间,并用注意力池化聚合选中时间戳,平衡时间特异性与鲁棒性。在三个MMKG基准上的实验表明,Time Imprint实现领先链接预测性能,整体Hits@1提升最高达6.07%,在最模糊的1%样本上提升高达58%。我们进一步分析不同融合策略及时间戳可用性与质量的敏感性,揭示时间作为模态的最佳适用场景,且训练开销仅小幅增加。代码已公开于https://anonymous.4open.science/r/Time-Imprint。
原文摘要 · Abstract (English)
Multi-Modal Knowledge Graphs (MMKGs) enrich entities with multiple modalities such as text and images, yet entities with highly similar multi-modal features remain difficult to distinguish. Temporal information of an entity can serve as an additional modality to disambiguate such entities, but existing approaches rarely treat time as a separate modality alongside text and images due to two major challenges: (1) sparse temporal semantics, which hinder alignment with richer modalities, and (2) multiple timestamps, which introduce noise or reduce robustness in representation learning. To address these challenges, we propose Time Imprint, a framework that treats time as an entity-level modality and jointly aligns temporal, textual, and visual representations via a three-view contrastive objective. Additionally, to mitigate multi-timestamp ambiguity, Time Imprint studies a compact timestamp subset selection design space and aggregates the selected timestamps into a discriminative temporal embedding with attention pooling, balancing temporal specificity and robustness. Experiments on three MMKG benchmarks demonstrate that Time Imprint achieves state-of-the-art link prediction performance, improving Hits@1 by up to 6.07\% overall and yielding up to 58\% gains on the subset of the top-1\% ambiguity samples. We further examine different fusion strategies and the sensitivity to timestamp availability and quality, clarifying when and why time-as-modality is most beneficial, while adding only modest training overhead. We release our code at https://anonymous.4open.science/r/Time-Imprint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。