提出可区分方向的神经发散头,提升语义关系建模精度
Role-Aware Neural Convex Divergence Heads for Asymmetric Representation Learning

- 通过角色投影+凸性Bregman发散,构建有向表征评分机制
- 在10个随机种子下方向准确率优于普通ICNN-Bregman头
- 适合需要精确方向建模的语义/本体任务,结果可解释性强
许多表示学习问题涉及有向关系,如词义蕴含、句子蕴含、本体层级和引用链接。标准欧氏、余弦和马哈拉诺比斯头具有对称性,而通用神经打分器虽能建模方向性,但几何结构有限。本文提出一种角色感知的神经凸发散头,用于非对称表示学习。该头在评估输入凸神经Bregman发散前,对源角色与目标角色分别进行投影,得到角色投影空间中的非负结构化得分。我们刻画了其投影空间恒等性、源角色凸性、方向间隙分解及基于海森矩阵的局部曲率。在词汇、句子、本体和有向图基准上进行实验,对比对称距离、无结构异向打分器、顺序/双曲基线、普通ICNN-Bregman头以及所提角色感知变体。在主要语义与本体基准上,10次随机种子测试中,角色感知投影始终优于普通ICNN-Bregman头,同时保持零负发散率。结果还发现边界情况:在大型固定特征引用预测任务中,专用对称或双曲基线在排序准确率上仍更优。总体而言,该头可视为方向关系重要任务中的结构化、可解释的即插即用距离模块。
原文摘要 · Abstract (English)
Many representation learning problems involve directed relations, such as lexical entailment, sentence entailment, ontology hierarchy, and citation links. Standard Euclidean, cosine, and Mahalanobis heads are symmetric, while generic neural scorers can model directionality but provide limited geometric structure. This paper proposes a role-aware neural convex divergence head for asymmetric representation learning. The head applies source- and target-role projections before evaluating an input-convex neural Bregman divergence, yielding a nonnegative structured score in the role-projected space. We characterize its projected-space identity, source-role convexity, directional-gap decomposition, and Hessian-based local curvature. Experiments on lexical, sentence, ontology, and directed graph benchmarks compare symmetric distances, unstructured asymmetric scorers, order/hyperbolic baselines, plain ICNN-Bregman heads, and the proposed role-aware variant. Across ten random seeds on the main semantic and ontology benchmarks, role-aware projections consistently improve directional accuracy over plain ICNN-Bregman heads while preserving zero observed negative divergence rate. The results also identify a boundary case: on large fixed-feature citation prediction, specialized symmetric or hyperbolic baselines remain stronger in ranking accuracy. Overall, the proposed head is best understood as a structured and interpretable plug-in distance module for tasks where directional relations matter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。