arXiv:2410.05935cs.CVcs.MM2024-10中稿 · ACM Multimedia Asi…被引 2

在特征空间加高斯噪声,提升漫画角色一键检测精度

Learning Gaussian Data Augmentation in Feature Space for One-shot Object Detection in Manga

  • 在特征空间添加可学习的高斯噪声增强查询图像
  • 在可见与不可见类别上均提升检测性能
  • 适合新角色频繁出现的漫画场景使用

我们针对日漫中的一键目标检测问题。随着日漫全球流行,角色面部检测在自动着色等应用中愈发重要,但受版权限制,难以获取足够训练数据。且每部新漫画都会引入新角色,重新训练检测器不现实。因此,仅需一张参考图即可检测新角色的一键检测至关重要。挑战在于目标图像中角色姿态和表情变化大,且角色出现频率呈长尾分布。为此,我们提出一种在特征空间进行高斯数据增强的方法:通过学习每个通道的噪声方差,对查询特征添加高斯噪声以增加变化。实验表明,该方法在可见与未见类别上均优于图像空间的数据增强方法。

原文摘要 · Abstract (English)

We tackle one-shot object detection in Japanese Manga. The rising global popularity of Japanese manga has made the object detection of character faces increasingly important, with potential applications such as automatic colorization. However, obtaining sufficient data for training conventional object detectors is challenging due to copyright restrictions. Additionally, new characters appear every time a new volume of manga is released, making it impractical to re-train object detectors each time to detect these new characters. Therefore, one-shot object detection, where only a single query (reference) image is required to detect a new character, is an essential task in the manga industry. One challenge with one-shot object detection in manga is the large variation in the poses and facial expressions of characters in target images, despite having only one query image as a reference. Another challenge is that the frequency of character appearances follows a long-tail distribution. To overcome these challenges, we propose a data augmentation method in feature space to increase the variation of the query. The proposed method augments the feature from the query by adding Gaussian noise, with the noise variance at each channel learned during training. The experimental results show that the proposed method improves the performance for both seen and unseen classes, surpassing data augmentation methods in image space.

一键检测特征增强漫画分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。