四类改进各有成本归属,资源有限团队应按部署阶段优先级采纳。
Where the Cost Falls: A Deployment-Aware Adoption Order for Stability Enhancements to Cycle-Consistent Adversarial Networks

- 增强方法分训练期与部署期两类,仅自注意力组件影响推理内存。
- 自注意力使内存随特征图尺寸平方增长,是唯一需延迟采用的模块。
- 适合计算或延迟受限团队参考,指导模型优化的落地顺序。
在无配对图像到图像翻译中,循环一致性对抗网络常面临训练震荡或崩溃、粗略布局保留而细粒度纹理漂移、全局判别器忽略局部瑕疵等问题。四项改进分别针对这些缺陷:带梯度惩罚的Wasserstein目标、基于VGG19的循环重建感知损失、多尺度判别器,均仅影响训练过程,可灵活启用或关闭而不影响部署版本;而自注意力机制则持续存在于生成器中,其内存开销随特征图大小的平方增长。通过在马到斑马转换任务上逐步集成这四项技术,我们验证了每项改进的目标失败模式及其融合方式。实验显示基线模型存在崩溃和重构伪影问题,可视化样本与FID、KID指标表明联合模型性能提升。研究呼吁建立统一评估协议,涵盖各变体、感知相似性及下游分割任务,以实证排序这些增强效果。
原文摘要 · Abstract (English)
Teams that adopt cycle-consistent adversarial networks for unpaired image-to-image translation meet the same obstacles: adversarial training oscillates or collapses, cycle consistency preserves coarse layout while finer texture drifts, and a single discriminator judging global realism misses local artifacts. Four enhancements address these failures, and they are usually compared on output quality alone. We show that they also divide sharply by where their cost falls, and that this division, which follows from the architecture and not from any particular run, yields an adoption order for teams under a compute or latency budget. A Wasserstein objective with gradient penalty, a VGG19 perceptual loss on the cycle reconstruction, and multi-scale discriminators change training only, so a team can adopt or drop them without altering what ships. Self-attention alone persists into the deployed generator, with memory growing as the square of the feature-map size, which makes it the one component a resource-constrained team should defer. We integrate all four onto a lightly tuned baseline for horse-to-zebra translation, introduced one at a time on a fixed control and then combined, and for each we give the failure mode it targets and how it integrates. We document the collapse and reconstruction-artifact modes the baseline produced, report what visual inspection of saved samples showed for each variant, and report Fréchet Inception Distance and Kernel Inception Distance for the combined model. We specify the protocol still needed, covering the individual variants, perceptual similarity, and downstream segmentation, to rank these enhancements on measured evidence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。