让AI像画家一样先构思、再画草图、最后上色,实现可控图像生成。
GenClaw: Code-Driven Agentic Image Generation

- 用代码(SVG/ThreeJS)生成可执行的视觉草图作为中间步骤
- 通过分阶段流程实现从概念到图像的可控生成
- 适合需要精确控制图像结构与内容的研究者和创作者
图像生成模型已从文本驱动的像素合成发展为具备视觉理解与工具调用能力的多模态智能体。然而,现有智能体仍受制于底层黑箱图像模型,工作流陷入反复改写提示以优化生成结果的循环,缺乏直接操作画面的能力。本质上,大语言模型作为真正‘画笔’进行精准视觉构建的潜力尚未被释放。本文提出 GenClaw,一种由代码驱动的智能体图像生成范式,使智能体能像人类艺术家一样:先构思,再绘草图,最后着色。具体而言,智能体首先通过搜索与推理构建概念知识与上下文;随后利用代码(如 SVG、HTML、ThreeJS)生成可执行的视觉草图;最后借助图像生成模型补充纹理、材质与写实效果。在此流程中,代码作为可控的中间画布,连接语言推理与像素合成,无缝融合程序逻辑与生成模型的视觉表现力。通过将图像生成从黑箱模式转变为类人创作的分阶段过程,GenClaw为高度可控且可解释的视觉生成系统迈出关键一步。
原文摘要 · Abstract (English)
Image generation models have evolved from text-conditioned pixel synthesis toward multimodal agents endowed with visual comprehension and tool invocation capabilities. Yet, existing agents remain at the mercy of underlying black-box image models. Their workflow is trapped in a repetitive cycle of prompt rewriting for generation refinement, leaving them with no mechanism to directly manipulate the canvas. In essence, the potential of LLMs to serve as a genuine "brush" for precise visual construction remains largely untapped. In this paper, we propose GenClaw, a code-driven agentic image generation paradigm that empowers the agent to create like a human artist: first conceptualizing, then sketching, and finally coloring. Specifically, the agent first constructs the conceptual knowledge and context through search and reasoning. It then utilizes code (e.g., SVG, HTML, ThreeJS) to render executable visual sketches. Finally, it employs an image generation model to supplement textures, materials, and photorealism. In this workflow, code serves as a controllable intermediate canvas bridging linguistic reasoning and pixel synthesis, seamlessly integrating programmatic logic with the visual expressiveness of generative models. By transforming image generation from a black-box paradigm into a staged process akin to authentic human creation, GenClaw offers a step toward for highly controllable and interpretable visual generation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。