arXiv:2609.02796cs.CL2026-09

让机器翻译更懂语篇,提升手语转换的连贯性。

DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation

  • 基于大模型模块化框架,处理空间指代、问答结构等语篇现象。
  • 在语篇数据集上显著提升实体空间一致性与追踪能力。
  • 首个系统性手语转写语篇框架,适合手语技术研究者使用。

手语处理系统传统上仅在句子层面运行,忽略了对理解手语至关重要的语篇现象。我们提出DiscoSign,一种基于语言学研究的语篇感知文本到手语词汇的计算方法。在基于大模型的模块化框架中,我们解决了三个关键现象:(i) 空间指代消解,即实体在整个语篇中保持一致的空间位置;(ii) 问答从句(QACs),一种具有特定语篇功能的伪分裂结构;(iii) 概念-词汇一致性,确保英语概念与美国手语(ASL)符号之间稳定映射。传统翻译评估指标无法捕捉语篇级质量,因此我们引入一套新评估指标,用于衡量框架所解决的每个语篇连贯维度。在句子级和语篇级数据集上的实验表明,该语篇感知处理方法相比仅基于句子的翻译,在空间一致性与实体追踪方面有显著提升,同时保持了具有竞争力的单句词汇翻译质量。本工作建立了首个系统性的语篇级文本到手语词汇转换框架及其相应的评估方法。

原文摘要 · Abstract (English)

Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language comprehension. We introduce DiscoSign, a computational approach for discourse-aware text to sign language gloss translation grounded in linguistic research. We address three key phenomena within our modular Large Language Model (LLM)-based translation framework: (i) spatial coreference resolution, where entities maintain consistent spatial locations throughout discourse; (ii) Question-Answer Clauses (QACs), pseudocleft structures serving specific discourse functions; and (iii) concept-gloss consistency, ensuring stable mappings between English concepts and American Sign Language (ASL) signs. Traditional translation metrics fail to capture discourse-level quality, so we introduce a suite of novel evaluation metrics designed to assess each dimension of discourse coherence addressed by our framework. Experiments on sentence-level and discourse-level datasets show that our approach for discourse-aware processing significantly improves spatial consistency and entity tracking relative to sentence-only translation, while maintaining competitive single-sentence gloss translation quality. Our work establishes the first systematic framework for discourse-level text to sign language gloss translation with corresponding evaluation methodology.

手语生成语篇分析大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。