用AI自动生成警察通缉画像，提升准确度与清晰度。
Gen-AI Police Sketches with Stable Diffusion
- 结合CLIP与Stable Diffusion，通过LoRA微调注意力层提升文本图像对齐。
- 基线模型结构相似度达0.72，峰值信噪比25 dB，表现最优。
- 适合刑侦人员快速生成高清晰度嫌疑人画像，辅助破案。
本项目研究多模态AI方法在自动化和优化嫌疑人画像中的应用。构建并评估了三种流程：(1) 基线图像到图像的Stable Diffusion模型；(2) 集成预训练CLIP模型以实现文本-图像对齐的相同模型；(3) 新型方法，在Stable Diffusion中引入对CLIP模型自注意力与交叉注意力层的LoRA微调。消融实验表明，同时微调两类注意力层能获得最佳的文本与图像对齐效果。性能测试显示，模型1在结构相似度（SSIM）上达到0.72，峰值信噪比（PSNR）为25 dB，优于模型2与模型3。迭代优化提升了感知相似度（LPIPS），其中模型3优于模型2，但仍落后于模型1。定性分析显示，模型1生成的画像面部特征最清晰，证明其虽简单却具有强鲁棒性。
原文摘要 · Abstract (English)
This project investigates the use of multimodal AI-driven approaches to automate and enhance suspect sketching. Three pipelines were developed and evaluated: (1) baseline image-to-image Stable Diffusion model, (2) same model integrated with a pre-trained CLIP model for text-image alignment, and (3) novel approach incorporating LoRA fine-tuning of the CLIP model, applied to self-attention and cross-attention layers, and integrated with Stable Diffusion. An ablation study confirmed that fine-tuning both self- and cross-attention layers yielded the best alignment between text descriptions and sketches. Performance testing revealed that Model 1 achieved the highest structural similarity (SSIM) of 0.72 and a peak signal-to-noise ratio (PSNR) of 25 dB, outperforming Model 2 and Model 3. Iterative refinement enhanced perceptual similarity (LPIPS), with Model 3 showing improvement over Model 2 but still trailing Model 1. Qualitatively, sketches generated by Model 1 demonstrated the clearest facial features, highlighting its robustness as a baseline despite its simplicity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。