为本地实时字幕翻译优化模型，提升速度与隐私保护。
Workload-Driven Optimization for On-Device Real-Time Subtitle Translation
- 用64k专用字典替换原151k字典，降低解码开销。
- 在OpenSubtitles2024上胜过谷歌翻译59.2%（无平局）。
- 适合短字幕场景，本地部署，低延迟高隐私。
本研究针对台湾地区短输入、短输出、单批推理、低延迟和隐私约束下的设备端英译繁体字幕任务展开。传统长上下文或高吞吐量优化方案在此不适用。从LMT-60-0.6B模型出发，初步分析发现，在GGUF量化后，词汇投影的解码成本相对高于Transformer块。为此，将原151k词表替换为64k字幕领域专用词表，迁移嵌入空间，并通过嵌入校准加全监督微调适配模型。在OpenSubtitles2024测试集上，LocalSubs在GPT-4o成对评估中取得59.2%的无平局胜率。性能在短句上最强，随句长增加而下降。另在Apple M2 Metal平台上预估显示，相比151k词表基线，提速1.63倍。代码已开源于https://github.com/aiden1020/localsubs。
原文摘要 · Abstract (English)
This report studies on-device English-to-Traditional-Chinese subtitle translation for Taiwan under short inputs, short outputs, batch-size-one inference, low latency, and privacy constraints. These conditions limit the value of optimizations designed for long-context or high-throughput language-model serving. Starting from LMT-60-0.6B, preliminary profiling suggests that vocabulary projection becomes a more important decode-time cost after GGUF quantization reduces the relative cost of Transformer blocks. We replace the original 151k-token vocabulary with a 64k-token subtitle-domain tokenizer, migrate the embedding space, and adapt the model through embedding calibration followed by full supervised fine-tuning. On an OpenSubtitles2024 test set, LocalSubs achieves a 59.2% tie-excluded win rate against Google Translate under GPT-4o pairwise judging. Performance is strongest on short cues and declines as cue length increases. In a separate preliminary Apple M2 Metal profiling run, LocalSubs shows a 1.63x speedup over a 151k-vocabulary baseline. The code is available on https://github.com/aiden1020/localsubs .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。