arXiv:2604.20357cs.CVcs.CL2026-04被引 1

统一手语数据预处理流程,让不同来源的视频能高效转化为训练可用格式。

SignDATA: Data Pipeline for Sign Language Translation

  • 通过配置驱动实现数据采集、裁剪、关键点提取等全流程标准化
  • 支持姿态与视频两种输出模式,可灵活切换不同后端模型
  • 公开可复现,便于对比不同处理策略对模型性能的影响

手语数据集因标注规范、片段时间、拍摄视角和隐私限制差异大,难以统一预处理。现有工作多只报告下游模型结果,而将原始视频转为训练可用的姿态或视频数据的预处理流程分散、依赖特定后端且文档薄弱。本文提出 SignDATA,一个基于配置的预处理工具包,可将异构手语语料库标准化为可比输出。系统提供两个端到端流程:姿态流程包含采集、定位、裁剪、关键点提取、归一化及 WebDataset 导出;视频流程以签名人裁剪视频替代姿态提取。SignDATA 提供通用接口,支持 MediaPipe 与 MMPose 两种后端互换,具备类型化任务配置、实验级覆盖、阶段检查点及基于配置与清单的哈希校验。通过面向研究的评估设计,验证了工具在后端对比、预处理消融实验以及隐私敏感视频生成方面的有效性。贡献在于构建了一个可复现的手语研究预处理层,使提取器选择、归一化策略与隐私权衡变得透明、可配置且可实证比较。代码开源:https://github.com/balaboom123/signdata-slt。

原文摘要 · Abstract (English)

Sign-language datasets are difficult to preprocess consistently because they vary in annotation schema, clip timing, signer framing, and privacy constraints. Existing work usually reports downstream models, while the preprocessing pipeline that converts raw video into training-ready pose or video artifacts remains fragmented, backend-specific, and weakly documented. We present SignDATA, a config-driven preprocessing toolkit that standardizes heterogeneous sign-language corpora into comparable outputs for learning. The system supports two end-to-end recipes: a pose recipe that performs acquisition, manifesting, person localization, clipping, cropping, landmark extraction, normalization, and WebDataset export, and a video recipe that replaces pose extraction with signer-cropped video packaging. SignDATA exposes interchangeable MediaPipe and MMPose backends behind a common interface, typed job schemas, experiment-level overrides, and per-stage checkpointing with config- and manifest-aware hashes. We validate the toolkit through a research-oriented evaluation design centered on backend comparison, preprocessing ablations, and privacy-aware video generation on datasets. Our contribution is a reproducible preprocessing layer for sign-language research that makes extractor choice, normalization policy, and privacy tradeoffs explicit, configurable, and empirically comparable.Code is available at https://github.com/balaboom123/signdata-slt.

手语识别数据管道姿态估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。