通过多尺度融合提升物体表征，让模型更好处理不同大小的物体。
Multi-Scale Fusion for Object Representation
- 构建图像金字塔，生成多尺度中间表示
- 跨尺度融合高低质量物体超像素，提升表征质量
- 在主流基准上超越现有方法，适合物体尺寸多变的任务
将图像或视频表示为物体级特征向量，而非像素级特征图，有助于推进高级视觉任务。物体中心学习（OCL）主要通过变分自编码器（VAE）中间表示引导输入重构，使所谓的“槽”尽可能聚合物体信息。然而，现有VAE引导未显式解决物体像素尺寸差异问题，而模型通常仅擅长特定尺度模式。为此，我们提出多尺度融合（MSF）以增强VAE对OCL训练的引导作用。为确保所有尺寸的物体均处于VAE的适配范围，我们采用图像金字塔，生成多尺度中间表示；为增强物体超像素的尺度不变性/变异性，设计了跨尺度与同尺度融合机制，用另一尺度的高质量超像素补充当前尺度的低质量超像素。在标准OCL基准上，该方法显著提升主流方法性能，包括最先进的基于扩散模型的方法。代码已开源：https://github.com/Genera1Z/MultiScaleFusion。
原文摘要 · Abstract (English)
Representing images or videos as object-level feature vectors, rather than pixel-level feature maps, facilitates advanced visual tasks. Object-Centric Learning (OCL) primarily achieves this by reconstructing the input under the guidance of Variational Autoencoder (VAE) intermediate representation to drive so-called \textit{slots} to aggregate as much object information as possible. However, existing VAE guidance does not explicitly address that objects can vary in pixel sizes while models typically excel at specific pattern scales. We propose \textit{Multi-Scale Fusion} (MSF) to enhance VAE guidance for OCL training. To ensure objects of all sizes fall within VAE's comfort zone, we adopt the \textit{image pyramid}, which produces intermediate representations at multiple scales; To foster scale-invariance/variance in object super-pixels, we devise \textit{inter}/\textit{intra-scale fusion}, which augments low-quality object super-pixels of one scale with corresponding high-quality super-pixels from another scale. On standard OCL benchmarks, our technique improves mainstream methods, including state-of-the-art diffusion-based ones. The source code is available on https://github.com/Genera1Z/MultiScaleFusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。