arXiv:2509.08489cs.CVcs.AI2025-09

一句话指令实现图像检测、分割、修复与描述,全流程透明可复现。

Prompt-Driven Image Analysis with Multimodal Generative AI: Detection, Segmentation, Inpainting, and Interpretation

  • 单个自然语言提示驱动多阶段图像分析流程
  • 90%以上场景生成准确率超85%的可用掩码
  • 支持调试可视化与可重复运行,适合工程落地

提示驱动的图像分析将单一自然语言指令转化为定位、分割、编辑和描述等多个步骤。本文展示了一个统一工作流的实用案例,整合了开放词汇检测、可提示分割、文本条件修复及视觉-语言描述功能。系统从单一提示端到端运行,保留中间结果(如检测框、掩码、叠加图、修复前后对比图)以实现透明调试,并通过交互式UI和可脚本化的命令行接口保证一致性和可重复性。研究强调降低脆弱性的集成策略,包括阈值调整、轻量形态学掩码检查及资源感知默认设置。在单字提示场景中,检测与分割在超过90%的情况下生成准确率高于85%的掩码;在高端GPU上,修复阶段占典型配置下总耗时的60%至75%,凸显调参重要性。论文提供阈值、掩码紧密度及扩散参数的实践建议,并详细说明版本锁定、结果日志与随机种子控制以支持复现。贡献在于构建一个透明可靠的多模态模型集成模式,具备明确约束与操作规范,适用于对象替换、场景增强与移除任务。

原文摘要 · Abstract (English)

Prompt-driven image analysis converts a single natural-language instruction into multiple steps: locate, segment, edit, and describe. We present a practical case study of a unified pipeline that combines open-vocabulary detection, promptable segmentation, text-conditioned inpainting, and vision-language description into a single workflow. The system works end to end from a single prompt, retains intermediate artifacts for transparent debugging (such as detections, masks, overlays, edited images, and before and after composites), and provides the same functionality through an interactive UI and a scriptable CLI for consistent, repeatable runs. We highlight integration choices that reduce brittleness, including threshold adjustments, mask inspection with light morphology, and resource-aware defaults. In a small, single-word prompt segment, detection and segmentation produced usable masks in over 90% of cases with an accuracy above 85% based on our criteria. On a high-end GPU, inpainting makes up 60 to 75% of total runtime under typical guidance and sampling settings, which highlights the need for careful tuning. The study offers implementation-guided advice on thresholds, mask tightness, and diffusion parameters, and details version pinning, artifact logging, and seed control to support replay. Our contribution is a transparent, reliable pattern for assembling modern vision and multimodal models behind a single prompt, with clear guardrails and operational practices that improve reliability in object replacement, scene augmentation, and removal.

图像生成多模态提示工程可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。