用检索代替分类，让多模态关系抽取更精准可解释
Retrieval over Classification: Integrating Relation Semantics for Multimodal Relation Extraction
- 将关系抽取转为基于语义的检索任务，融合实体类型与位置信息
- 在MNRE和MORE数据集上达到最新性能，提升鲁棒性与可解释性
- 适合需要细粒度关系理解与模型透明性的研究者使用
关系抽取（RE）旨在从非结构化文本中识别实体间的语义关系。尽管近期工作将传统RE拓展至多模态场景，但多数方法仍采用基于分类的范式，以融合的多模态特征表示关系为离散标签。该范式存在两大局限：(1) 忽视实体类型与位置线索等结构约束；(2) 缺乏对细粒度关系的语义表达能力。我们提出一种名为检索优于分类（ROC）的新框架，将多模态关系抽取重新定义为由关系语义驱动的检索任务。ROC通过多模态编码器整合实体类型与位置信息，利用大语言模型将关系标签扩展为自然语言描述，并通过基于语义相似性的对比学习对齐实体-关系对。实验表明，该方法在基准数据集MNRE和MORE上取得当前最优性能，且展现出更强的鲁棒性与可解释性。
原文摘要 · Abstract (English)
Relation extraction (RE) aims to identify semantic relations between entities in unstructured text. Although recent work extends traditional RE to multimodal scenarios, most approaches still adopt classification-based paradigms with fused multimodal features, representing relations as discrete labels. This paradigm has two significant limitations: (1) it overlooks structural constraints like entity types and positional cues, and (2) it lacks semantic expressiveness for fine-grained relation understanding. We propose \underline{R}etrieval \underline{O}ver \underline{C}lassification (ROC), a novel framework that reformulates multimodal RE as a retrieval task driven by relation semantics. ROC integrates entity type and positional information through a multimodal encoder, expands relation labels into natural language descriptions using a large language model, and aligns entity-relation pairs via semantic similarity-based contrastive learning. Experiments show that our method achieves state-of-the-art performance on the benchmark datasets MNRE and MORE and exhibits stronger robustness and interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。