arXiv:2510.23538cs.AIcs.CL2025-10被引 8

构建首个大规模多模态代码数据集,实现文本与视觉输入的统一编程生成。

JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence

  • 通过跨模态协同合成技术,生成覆盖图表到交互网页的80万条高质量多模态代码数据。
  • 训练出的JanusCoder模型在文本和视觉编码任务上表现超越现有方法,7B至14B规模接近商用水平。
  • 适合研究多模态编程、智能代码生成与可视化编辑的开发者与研究人员。

神经代码智能正从纯文本代码扩展至程序生成的丰富视觉输出。这一视觉维度对灵活内容生成和程序驱动的可视化编辑至关重要,但受限于高质量多模态代码数据稀缺,其发展受阻。为此,我们从数据与建模双角度提出贡献:首先开发一套完整的合成工具链,利用模态间互惠协同,高效生成涵盖标准图表、复杂交互网页及代码驱动动画的大规模高质量语料。基于该工具链,构建了目前最大规模的多模态代码语料库JanusCode-800K。该数据支撑训练出JanusCoder与JanusCoderV,建立统一的视觉-程序接口,支持从文本指令、视觉输入或二者结合生成代码。相比现有针对单一任务的专用模型,本工作实现一体化建模。大量实验表明,该系列模型在文本主导与视觉主导编码任务中均具优越性能,7B至14B规模模型已逼近甚至超过商用模型。深入分析揭示了程序逻辑与其视觉表达的协调机制。代码与模型权重已开源。

原文摘要 · Abstract (English)

The scope of neural code intelligence is rapidly expanding beyond text-based source code to encompass the rich visual outputs that programs generate. This visual dimension is critical for advanced applications like flexible content generation and precise, program-driven editing of visualizations. However, progress has been impeded by the scarcity of high-quality multimodal code data, a bottleneck stemming from challenges in synthesis and quality assessment. To address these challenges, we make contributions from both a data and modeling perspective. We first introduce a complete synthesis toolkit that leverages reciprocal synergies between data modalities to efficiently produce a large-scale, high-quality corpus spanning from standard charts to complex interactive web UIs and code-driven animations. Leveraging this toolkit, we construct JanusCode-800K, the largest multimodal code corpus to date. This powers the training of our models, JanusCoder and JanusCoderV, which establish a visual-programmatic interface for generating code from textual instructions, visual inputs, or a combination of both. Our unified model is a departure from existing approaches that build specialized models for isolated tasks. Extensive experiments on both text-centric and vision-centric coding tasks demonstrate the superior performance of the JanusCoder series, with our 7B to 14B scale models approaching or even exceeding the performance of commercial models. Furthermore, extensive analysis provides key insights into harmonizing programmatic logic with its visual expression. Our code and checkpoints are available at https://github.com/InternLM/JanusCoder.

代码生成多模态视觉编程大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。