通过人类干预反馈,让分割模型学会避开虚假特征,提升真实场景适应力。
Explainable Human-in-the-Loop Segmentation via Critic Feedback Signals
- 用人类修正作为干预信号,引导模型学习语义特征而非表面关联
- 在复杂立方图数据上提升9个mIoU点,相对改进12-15%
- 只需少量标注即可显著降低标注成本,适合实际应用落地
分割模型在基准测试中表现优异,但在真实场景中常因依赖虚假相关性(如颜色、纹理)而失败。本文提出一种人机协同的交互式框架,通过针对性的人类修正提供干预信号,揭示模型在何时何地误判。系统通过传播这些修正信息至视觉相似图像,引导模型学习更鲁棒、语义合理的特征,避免数据集特定偏差。相比传统重新标注方式,本方法能系统识别并纠正模型缺陷,实现跨数据集的泛化优化。在挑战性的cubemap数据集上,准确率最高提升9 mIoU(相对提升12-15%),同时标注效率提高3-4倍,且在标准基准上仍保持竞争力。该框架为城市气候监测、自动驾驶等真实场景中的高可靠性分割系统提供了实用解决方案。
原文摘要 · Abstract (English)
Segmentation models achieve high accuracy on benchmarks but often fail in real-world domains by relying on spurious correlations instead of true object boundaries. We propose a human-in-the-loop interactive framework that enables interventional learning through targeted human corrections of segmentation outputs. Our approach treats human corrections as interventional signals that show when reliance on superficial features (e.g., color or texture) is inappropriate. The system learns from these interventions by propagating correction-informed edits across visually similar images, effectively steering the model toward robust, semantically meaningful features rather than dataset-specific artifacts. Unlike traditional annotation approaches that simply provide more training data, our method explicitly identifies when and why the model fails and then systematically corrects these failure modes across the entire dataset. Through iterative human feedback, the system develops increasingly robust representations that generalize better to novel domains and resist artifactual correlations. We demonstrate that our framework improves segmentation accuracy by up to 9 mIoU points (12-15\% relative improvement) on challenging cubemap data and yields 3-4$\times$ reductions in annotation effort compared to standard retraining, while maintaining competitive performance on benchmark datasets. This work provides a practical framework for researchers and practitioners seeking to build segmentation systems that are accurate, robust to dataset biases, data-efficient, and adaptable to real-world domains such as urban climate monitoring and autonomous driving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。