用双阶段生成网络实时提升游戏画面真实感,帧率提升12倍。
REGEN: Real-Time Photorealism Enhancement in Games via a Dual-Stage Generative Network Framework
- 先用无配对模型生成语义一致的真实画面,再构建配对数据集
- 轻量级模型实现实时推理,视觉质量与原有方法相当或略优
- 保持画面语义一致性和时间连续性,适合游戏实时渲染
逼真度是现代视频游戏的关键,影响玩家沉浸感、叙事体验和视觉精度。除传统渲染流程外,生成模型正被广泛用于弥合合成世界与真实世界之间的视觉差距。然而,在游戏实时性约束下,现有生成方法仍面临视觉质量与运行效率的权衡。本文提出REGEN框架,通过双阶段生成网络增强游戏帧的逼真度:首先使用鲁棒的无配对图像到图像转换模型生成语义一致的逼真帧;再利用这些生成帧构建配对数据集,将问题转化为更简单的无配对转换任务,从而训练轻量级模型实现无需牺牲画质的实时推理。在Unreal Engine上的评估显示,采用CMMD指标,REGEN在视觉质量上达到或略优于基准方法,同时帧率提升12倍。额外实验验证了其在语义保留和时间一致性方面的稳定性。代码、预训练模型及演示已公开于https://github.com/stefanos50/REGEN。
原文摘要 · Abstract (English)
Photorealism is an important aspect of modern video games since it can shape player experience and impact immersion, narrative engagement, and visual fidelity. To achieve photorealism, beyond traditional rendering pipelines, generative models have been increasingly adopted as an effective approach for bridging the gap between the visual realism of synthetic and real worlds. However, under real-time constraints of video games, existing generative approaches continue to face a tradeoff between visual quality and runtime efficiency. In this work, we present a framework for enhancing the photorealism of rendered game frames using generative networks. We propose REGEN, which first employs a robust unpaired image-to-image translation model to generate semantically consistent photorealistic frames. These generated frames are then used to create a paired dataset, which transforms the problem to a simpler unpaired image-to-image translation. This enables training with a lightweight method, achieving real-time inference without compromising visual quality. We evaluate REGEN on Unreal Engine, showing, by employing the CMMD metric, that it achieves comparable or slightly improved visual quality compared to the robust method, while improving the frame rate by 12x. Additional experiments also validate that REGEN adheres to the semantic preservation of the initial robust image-to-image translation method and maintains temporal consistency. Code, pre-trained models, and demos for this work are available at: https://github.com/stefanos50/REGEN
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。