改进小目标图像分类的内存高效采样方法，提升低数据场景下的泛化能力。
On the Generalizability of Iterative Patch Selection for Memory-Efficient High-Resolution Image Classification
- 采用迭代补丁选择机制，在保持低内存消耗的同时优化关键区域捕获。
- 在极低数据量下，缩小补丁尺寸可使准确率提升15%（百万像素MNIST）和5%（瑞典交通标志）。
- 发现噪声厚度与数字特征相似时，模型易失效，适用于小样本、高分辨率图像任务。
在小或极小感兴趣区域（ROI）的大图像分类中，计算与内存限制带来挑战。弱监督的内存高效补丁选择器已达到与强监督方法相当的效果，但低信噪比和低熵注意力仍导致过拟合。本文通过在内存高效交叉注意力变压器中引入迭代补丁选择（IPS）模块，构建新测试基准，将百万像素MNIST基准扩展至四个更小的物体-图像比（O2I），范围为0.01%至0.14%，同时固定画布大小并引入基于贝塞尔曲线的噪声生成组件。实验表明，先前在CNN上的观察可推广至IPS：分类器泛化失败的临界O2I阈值受训练数据集大小影响。不同任务间该交互效应差异显著，其中‘Maj’和‘Top’任务最高，‘Max’和‘Multi’任务接近0。在低数据条件下，将补丁尺寸相对于ROI调小可改善泛化，使百万像素MNIST准确率提升+15%，瑞典交通标志数据集提升+5%。进一步结果表明，当噪声厚度与数字特征逐渐相似时，IPS逐渐失去泛化能力，支持此前关于其失效机制的猜想。
原文摘要 · Abstract (English)
Classifying large images with small or tiny regions of interest (ROI) is challenging due to computational and memory constraints. Weakly supervised memory-efficient patch selectors have achieved results comparable with strongly supervised methods. However, low signal-to-noise ratios and low entropy attention still cause overfitting. We explore these issues using a novel testbed on a memory-efficient cross-attention transformer with Iterative Patch Selection (IPS) as the patch selection module. Our testbed extends the megapixel MNIST benchmark to four smaller O2I (object-to-image) ratios ranging from 0.01% to 0.14% while keeping the canvas size fixed and introducing a noise generation component based on Bézier curves. Experimental results generalize the observations made on CNNs to IPS whereby the O2I threshold below which the classifier fails to generalize is affected by the training dataset size. We further observe that the magnitude of this interaction differs for each task of the Megapixel MNIST. For tasks "Maj" and "Top", the rate is at its highest, followed by tasks "Max" and "Multi" where in the latter, this rate is almost at 0. Moreover, results show that in a low data setting, tuning the patch size to be smaller relative to the ROI improves generalization, resulting in an improvement of + 15% for the megapixel MNIST and + 5% for the Swedish traffic signs dataset compared to the original object-to-patch ratios in IPS. Further outcomes indicate that the similarity between the thickness of the noise component and the digits in the megapixel MNIST gradually causes IPS to fail to generalize, contributing to previous suspicions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。