arXiv:2609.06843cs.CV2026-09

用CTC解码器定位手语关键帧,提升主动学习数据效率

RAIDAL: Redundancy-Aware Information Density Active Learning for CTC-Based Continuous Sign Language Recognition

论文配图:RAIDAL: Redundancy-Aware Information Density Active Learning for CTC-Based Continuous Sign Language Recognition
图 1 · 摘自论文原文
  • 利用CTC解码器对齐峰值定位手语词级区域
  • 在大词汇量小预算下比基线少用30%数据达同等精度
  • 无需额外标注,适配弱对齐手语视频场景

连续手语识别(CSLR)对无障碍技术至关重要,但高昂的视频标注成本限制了其发展。主动学习可降低标注成本,但传统方法不适用于弱对齐的手语视频——这些视频包含手语动作、静止姿态、不规则停顿及时间冗余帧。这些冗余会干扰样本选择,使评分受非目标时段影响,扭曲信息量估计。本文发现,现代CSLR模型中的CTC解码器能自动识别词级时间证据:其对齐峰值可定位每个预测词在特征序列中的位置,提供零额外标注成本的时序结构。为此提出RAIDAL(冗余感知信息密度主动学习),将基于表示的评分限制在解码器对齐的词级区域,而非整个未过滤视频。在三个数据集和两种架构上,RAIDAL在大词汇量、低预算场景中显著提升数据效率,而在小词汇量、高预算场景中保持竞争力。代码已开源。

原文摘要 · Abstract (English)

Continuous sign language recognition (CSLR) is a key technology for accessibility, yet its development remains limited by the high cost of annotating continuous video streams. Active learning offers a path toward mitigating this cost, but standard acquisition functions are not designed for weakly aligned sign language videos, where sign executions are interleaved with rest poses, irregular pauses, sign-like motion, and temporally redundant frames. This temporal redundancy can undermine sample selection, as acquisition scores may be influenced by timesteps from regions that are not associated with the decoded gloss sequence, distorting the video's estimated informativeness. In this work, we show that modern CSLR models already contain a mechanism for identifying gloss-level temporal evidence: the CTC decoder. Although typically used only during inference, its alignment peaks indicate where the model localizes each predicted gloss in the feature sequence, providing a source of temporal structure for active learning acquisition functions at zero additional labeling cost. Thus, we introduce RAIDAL (Redundancy-Aware Information Density Active Learning), which repurposes the CTC decoder to restrict representation-based scoring to decoder-aligned gloss regions, rather than exposing the acquisition function to the entire unfiltered video. Across three datasets and two architectures, RAIDAL achieves its strongest data-efficiency gains over competing baselines in large-vocabulary, budget-limited settings, while remaining competitive in the smaller-vocabulary, large-budget setting. The code used in this work is publicly available at github.com/verlab/RAIDAL.

主动学习手语识别CTC解码器数据效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。