精准控制人群图像中每个个体的位置与大小,生成逼真密集人群场景。
DenseControl: Instance-Level Controllable Synthesis of Dense Crowd Image

- 提出IOE地图和ISE策略,实现个体位置与尺度的精确控制。
- 在多种控制条件下,合成质量达当前最优水平。
- 适合需要高精度人群数据增强的研究者使用。
本文提出DenseControl,一种生成密集人群图像的新方法。该方法通过精确定位和缩放每个生成实例,使其严格匹配预设坐标与尺度。为解决信号嵌入控制与实例尺度引导下的拓扑完整性问题,我们引入孤立物体嵌入(IOE)图,提升空间位置控制能力;并提出隐式尺度嵌入(ISE)策略,无缝融合于IOE图以编码精确尺度信息。为进一步优化两者结合效果,设计位置捷径机制,增强跨注意力以缓解投影难题。在合成质量与潜在应用双重评估下,DenseControl在不同控制条件下均达到领先性能。实验展示其在数据稀缺、迁移学习与天气泛化等场景下增强人群分析的有效性。代码将公开。
原文摘要 · Abstract (English)
In this paper, we introduce DenseControl, a novel pipeline for generating dense crowd images. Specifically, DenseControl meticulously positions and sizes each generated instance to align precisely with the predefined coordinates and scales. Based on this, we further allow for control over the background, style, and attributes of instances. The motivation behind DenseControl stems from the observation of two main challenges in synthesizing crowd images: controlling signal embedding and maintaining topological integrity when imparting instance scale guidance. To address these, we first introduce the Isolated Object Embedding (IOE) map, a novel representation that facilitates spatial location control while mitigating the difficulties associated with learning projections for model. Secondly, we propose an Implicit Scale Embedding (ISE) strategy that seamlessly integrates with the IOE map to encode precise scale information. To further enhance the efficacy of combining ISE with the IOE map, we incorporate a Position Shortcut mechanism that enhances cross-attention to alleviate projection challenges. We evaluate DenseControl through two lenses: synthesis quality and applicability in latent applications. Experiments across different control conditions demonstrate DenseControl achieves state-of-the-art results in dense crowd image synthesis. Furthermore, we showcase applications in augmenting crowd analysis under data scarcity, transfer learning, and weather generalization scenes, to highlight the practical utility of DenseControl. The codebase will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。