用双曲几何建模视觉与脑神经的层次关系，提升跨模态对齐效果。
HyNeuralMap: Hyperbolic Mapping of Visual Semantics to Neural Hierarchies
- 采用双曲洛伦兹模型构建跨被试神经层次结构
- 在多标签预测与跨模态检索中优于欧式基线
- 适合研究视觉-神经映射与层次语义建模的学者
理解视觉刺激与神经反应之间的复杂映射是认知神经科学的核心挑战。现有方法多在欧几里得空间中对齐图像与功能磁共振成像（fMRI）响应，但该几何结构难以保持细粒度语义关系和跨视觉与神经模态的潜在层次结构。为此，我们提出HyNeuralMap，一种利用双曲洛伦兹模型将视觉语义映射到共享跨被试神经层次的框架。通过引入双曲空间的负曲率作为归纳偏置，该框架更有效地捕捉层次化语义组织与跨被试神经相似性。具体而言，视觉与神经嵌入通过双曲几何对齐联合优化，测地线距离能更准确地保留语义邻近性和层级关系。实验表明，HyNeuralMap在多标签语义预测与跨模态检索任务中持续优于最先进的欧氏基线。这证实了双曲几何在跨模态语义对齐与层次建模中的优势，为视觉-神经表征学习提供了新路径。
原文摘要 · Abstract (English)
Understanding the intricate mappings between visual stimuli and neural responses is a fundamental challenge in cognitive neuroscience. While current approaches predominantly align images and functional magnetic resonance imaging (fMRI) responses in Euclidean space, this geometry often struggles to preserve fine-grained semantic relationships and latent hierarchical structures across visual and neural modalities. To overcome this, we propose HyNeuralMap, a framework that employ hyperbolic Lorentz model to map visual semantics into a shared, cross-subject neural hierarchy. By leveraging the negative curvature of hyperbolic space as an inductive bias, the proposed framework better captures hierarchical semantic organization and cross-subject neural similarities. Specifically, visual and neural embeddings are jointly optimized through hyperbolic geometric alignment, where geodesic distances preserve semantic proximity and hierarchical relationships more effectively than Euclidean embeddings. Experiments demonstrate that HyNeuralMap consistently outperforms state-of-the-art Euclidean baselines in both multi-label semantic prediction and cross-modal retrieval tasks. This confirms hyperbolic geometry's superiority for cross-modal semantic alignment and hierarchical modeling, providing a new avenue for vision-neural representation learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。