用双曲几何优化手语骨骼表示，提升细微动作识别精度。
Geo-Sign: Hyperbolic Contrastive Regularisation for Geometrically Aware Sign Language Translation
- 将骨骼特征投影到双曲空间，捕捉手语的层次结构
- 在ST-GCN基础上提升手语翻译准确率，优于当前最先进方法
- 适合关注手语识别、几何建模与低资源场景的研究者
手语翻译研究近年聚焦于增强大语言模型对手语特征的表征能力。本文探索另一方向：直接提升骨骼表示的几何属性。提出Geo-Sign方法，利用双曲几何建模手语运动中固有的层次结构。通过将时空图卷积网络（ST-GCN）提取的骨骼特征投影至庞加莱球模型，旨在生成更具区分性的嵌入，尤其适用于细粒度动作如手指运动。引入双曲投影层、加权弗雷歇均值聚合方案及直接在双曲空间中运行的几何对比损失。这些组件作为正则化函数集成于端到端翻译框架，增强语言模型中的表示能力。实验表明，该方法在保持隐私和计算效率的同时，超越现有基于RGB的方法，达到新的性能基准。代码已开源：https://github.com/ed-fish/geo-sign。
原文摘要 · Abstract (English)
Recent progress in Sign Language Translation (SLT) has focussed primarily on improving the representational capacity of large language models to incorporate Sign Language features. This work explores an alternative direction: enhancing the geometric properties of skeletal representations themselves. We propose Geo-Sign, a method that leverages the properties of hyperbolic geometry to model the hierarchical structure inherent in sign language kinematics. By projecting skeletal features derived from Spatio-Temporal Graph Convolutional Networks (ST-GCNs) into the Poincaré ball model, we aim to create more discriminative embeddings, particularly for fine-grained motions like finger articulations. We introduce a hyperbolic projection layer, a weighted Fréchet mean aggregation scheme, and a geometric contrastive loss operating directly in hyperbolic space. These components are integrated into an end-to-end translation framework as a regularisation function, to enhance the representations within the language model. This work demonstrates the potential of hyperbolic geometry to improve skeletal representations for Sign Language Translation, improving on SOTA RGB methods while preserving privacy and improving computational efficiency. Code available here: https://github.com/ed-fish/geo-sign.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。