将电商图像文字翻译转化为可编辑的结构化代码生成
TransAnyText: Translating Arbitrary Text in E-commerce Images via Structured Visual Generation

- 用HTML片段替代像素级生成,分离语义与渲染
- 三阶段训练提升翻译准确率与视觉一致性
- 专为跨境电商业务设计,支持多语言可编辑输出
跨境电商业务中,商品图、横幅和详情页需多语言适配,现有方法难以同时保证翻译准确性、视觉身份一致性和可编辑性。为此,我们提出TransAnyText,一种基于结构化视觉代码的框架,将图像文本翻译重构为从源图和目标语言生成可渲染HTML片段的任务。该框架解耦语义生成与像素渲染:视觉语言模型(VLM)负责理解视觉内容、跨语言翻译及结构化生成;扩散模型则进行背景修复与像素级优化,再通过确定性渲染合成最终图像。基于此,我们构建了三阶段后训练框架:监督微调(SFT)建立图像到代码映射,特权差距加权自蒸馏(PWSD)优化风格与布局令牌学习,基于可验证奖励的强化学习(RLVR)进一步提升任务性能。我们还推出了TransAnyDataset和TransAnyBench,一个多语言数据集与基准测试。大量实验表明,其表现优于级联式流程、开源端到端模型及闭源图像编辑系统,为跨境电商品提供高效、可控且可编辑的解决方案。
原文摘要 · Abstract (English)
Cross-border e-commerce image translation is essential for global retail, where product images, banners, and detail pages need to be produced in different languages. Existing methods struggle to achieve accurate translation, faithful visual identity preservation, and easy-to-edit outputs, simultaneously. To address these challenges, we introduce TransAnyText, a structured visual code framework that reformulates image text translation as generating renderable HTML patches from source images and target languages. Our framework decouples semantic generation from pixel rendering: a vision-language model (VLM) handles visual understanding, cross-lingual translation, and structured visual generation, while a diffusion model performs background inpainting and pixel-level refinement, followed by deterministic rendering to synthesize the final image. Based on this formulation, we develop a three-stage post-training framework, where supervised fine-tuning (SFT) establishes the image-to-code mapping, privilege-gap weighted self-distillation (PWSD) improves the learning of style and layout tokens, and reinforcement learning with verifiable rewards (RLVR) further optimizes task-level performance. We further introduce TransAnyDataset and TransAnyBench, a multilingual dataset and benchmark for e-commerce image translation. Extensive experiments demonstrate competitive performance against cascaded pipelines, open-source end-to-end models, and closed-source image editing systems, providing an effective, controllable, and editable solution for cross-border e-commerce image translation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。