自动将2D游戏角色图转为可骨骼动画的三角网格,3秒完成,效率提升300倍。
SPRITETOMESH: Automatic Mesh Generation for 2D Skeletal Animation Using Learned Segmentation and Contour-Aware Vertex Placement
- 先用深度学习分割图像,再用算法精准定位轮廓和内部顶点。
- 分割准确率IoU达0.87,生成的网格支持Spine2D等动画框架。
- 适合游戏开发者,尤其需批量处理角色动画的团队。
我们提出SPRITETOMESH,一个完全自动化的流程,将2D游戏精灵图转换为兼容Spine2D等骨骼动画框架的三角网格。传统方式需人工逐个放置顶点,每张图耗时15-60分钟。本方法采用混合学习-算法策略:基于172款游戏的超10万对精灵-掩码数据训练的EfficientNet-B0编码器+U-Net解码器分割网络,取得0.87的IoU;从掩码中通过自适应弧分段的Douglas-Peucker简化提取外轮廓顶点,结合双边滤波多通道Canny边缘检测与轮廓追踪定位内顶点;最后经带掩码中心点过滤的Delaunay三角剖分生成网格。控制实验表明,直接用神经网络预测顶点位置不可行(损失稳定在0.061),因顶点布局具有艺术多样性。该负结果验证了混合设计的合理性:学习部分用于清晰标注,算法部分利用领域启发式。整个流程单图处理时间低于3秒,相较人工快300至1200倍。模型已开源供游戏开发社区使用。
原文摘要 · Abstract (English)
We present SPRITETOMESH, a fully automatic pipeline for converting 2D game sprite images into triangle meshes compatible with skeletal animation frameworks such as Spine2D. Creating animation-ready meshes is traditionally a tedious manual process requiring artists to carefully place vertices along visual boundaries, a task that typically takes 15-60 minutes per sprite. Our method addresses this through a hybrid learned-algorithmic approach. A segmentation network (EfficientNet-B0 encoder with U-Net decoder) trained on over 100,000 sprite-mask pairs from 172 games achieves an IoU of 0.87, providing accurate binary masks from arbitrary input images. From these masks, we extract exterior contour vertices using Douglas-Peucker simplification with adaptive arc subdivision, and interior vertices along visual boundaries detected via bilateral-filtered multi-channel Canny edge detection with contour-following placement. Delaunay triangulation with mask-based centroid filtering produces the final mesh. Through controlled experiments, we demonstrate that direct vertex position prediction via neural network heatmap regression is fundamentally not viable for this task: the heatmap decoder consistently fails to converge (loss plateau at 0.061) while the segmentation decoder trains normally under identical conditions. We attribute this to the inherently artistic nature of vertex placement - the same sprite can be meshed validly in many different ways. This negative result validates our hybrid design: learned segmentation where ground truth is unambiguous, algorithmic placement where domain heuristics are appropriate. The complete pipeline processes a sprite in under 3 seconds, representing a speedup of 300x-1200x over manual creation. We release our trained model to the game development community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。