顶尖图像生成模型正制造逼真假图,威胁社会信任根基。
Seeing Is No Longer Believing: Frontier Image Generation Models, Synthetic Visual Evidence, and Real-World Risk
- 用真实文本+高保真图像生成可信假证据
- 风险源于真实感、可读文字与快速传播的结合
- 适合政策制定者、媒体、金融和医疗从业者关注
前沿图像生成已从艺术创作转向合成视觉证据。GPT Image 2、Nano Banana Pro 等模型具备逼真渲染、可读文字、参考一致性、编辑控制能力,部分还支持推理或搜索驱动构建。这些能力虽提升设计、教育、沟通效率,却削弱了“看得像就可信”的社会信任基础。本文分析公开模型能力及多起虚假危机图、名人影像、医学影像、伪造文件、合成截图、钓鱼资产与市场谣言事件。提出能力加权风险框架,将模型特性与金融、医疗、新闻、法律等领域的现实危害关联。研究发现,风险不单来自逼真度,而在于真实感、可读文本、身份一致性、快速迭代与传播语境的叠加。主张分层管控:模型限制、加密溯源、可见标记、平台摩擦、行业认证与应急响应。最后向模型提供方、平台、媒体、金融机构、医疗机构、法律组织、监管机构及普通用户提出具体建议。
原文摘要 · Abstract (English)
Frontier image generation has moved from artistic synthesis toward synthetic visual evidence. Systems such as GPT Image 2, Nano Banana Pro, Nano Banana 2, Nano Banana 2 Lite, Grok Imagine Image Quality, Qwen Image 2.0 Pro, and Seedream 5.0 Lite combine photorealistic rendering, readable typography, reference consistency, editing control, and in several cases reasoning or search-grounded image construction. These capabilities create large benefits for design, education, accessibility, and communication, yet they also weaken one of society's most common trust shortcuts: the belief that a plausible picture is a reliable record. This paper provides a source-grounded technical and policy analysis of synthetic visual risk. We first summarize the public capabilities of recent image models, then analyze public incidents involving fake crisis images, celebrity and public-figure imagery, medical scans, forged-looking documents, synthetic screenshots, phishing assets, and market-moving rumors. We introduce a capability-weighted risk framework that links model affordances to real-world harm in finance, medicine, news, law, emergency response, identity verification, and civic discourse. Our findings show that risk is driven less by photorealism alone than by the convergence of realism, legible text, identity persistence, fast iteration, and distribution context. We argue for layered control: model-side restrictions, cryptographic provenance, visible labeling, platform friction, sector-grade verification, and incident response. The paper closes with practical recommendations for model providers, platforms, newsrooms, financial institutions, healthcare systems, legal organizations, regulators, and ordinary users.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。