用客户端遮罩合成,就能让美颜编辑只改指定部位。
Localize, Don't Beautify: Client-Side Control of Image-Editing APIs for Cosmetic Surgery Previews
- 用关键点生成遮罩,把修改区域裁剪回原图,实现精准定位。
- 遮罩合成使修改集中在目标区域,中位提升0.446,身份保留更好。
- 无需模型内部接口,适合想控制编辑范围的开发者或用户。
请求商业图像编辑器预览整形手术效果时,常会无意改变面部其他区域(如鼻部调整可能顺带平滑皮肤或改变光照)。现有方法需访问模型内部,但公开编辑API不提供此权限。本文研究仅在客户端能否实现精确控制。在初步评测中,六款商用编辑配置与一种基于掩码的修复模型,在三种客户端控制级别下完成12张正面人脸的面部轮廓和鼻部整形编辑:仅凭提示词;用关键点生成的掩码将修改区域裁剪并贴回原图(掩码合成);在支持的情况下由模型自身在掩码内修复。共210次尝试,196次可评分。使用ArcFace余弦度量身份保留,CIELAB像素变化比率衡量修改是否集中在目标区域而非保护面部区域。在12张可评分的人脸上,掩码合成相比仅提示词输出,中位定位精度提升0.446(95%置信区间0.421–0.457),且对目标区域改动程度相当。各编辑器在修改强度与身份保留间表现各异,测试的唯一修复模型未优于简单合成。所有编辑结果均未比输入更接近术后照片的身份嵌入表示。本研究聚焦控制能力,非临床准确性:无医生评估输出,每种条件仅生成一次。结论是:只要用遮罩与合成,即可在不访问模型的前提下,确保整形预览仅作用于目标区域,且对服务端成本极低。
原文摘要 · Abstract (English)
Ask a commercial image editor to preview a cosmetic procedure and it will often change more of the face than the request names: a nose edit can also smooth skin or alter lighting. Existing methods for confining an edit to one region require access to the model's internals, which a public editing API does not expose. We ask how much control is possible from the client side alone. In a pilot benchmark, six commercial editing configurations and one mask-based inpainting model perform facelift-style jaw-neck and rhinoplasty edits at three levels of client-side control: the prompt alone; cutting the edited region out of the response and pasting it back onto the original photograph through a landmark-derived mask (a masked composite); and asking the model itself to inpaint inside the mask where supported. Of 210 attempted edits, 196 could be scored. ArcFace cosine measures identity preservation; a CIELAB pixel-change ratio measures how much change lands inside the requested region rather than a protected facial zone. On the 12 frontal faces the regional metric could score, the masked composite improved localization over the paired prompt-only output by a median of 0.446 (95% face-clustered bootstrap interval 0.421-0.457) while changing the requested region about as much. Editors differed in edit strength versus identity retention, and the one inpainting model we tested did not beat the simple composite. Against each face's input-to-postoperative baseline, no editor moved its outputs closer to the postoperative photograph in identity-embedding terms. This is a study of control, not clinical accuracy: no surgeons rated the outputs, and each condition was generated once. Within that scope, keeping a surgical preview inside its intended region needs no access to the model; a mask and composite on the client enforce it across every editor tested, at low provider cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。