提出无参考评估方法,发现未微调模型反而更擅长遥感图像描述。
Evaluating Remote Sensing Image Captions Beyond Metric Biases

- 用文本重建视觉元素替代相似度计算,消除人工标注偏差。
- 未微调的大模型在零样本任务中表现超越微调版本。
- 无需训练即可提升生成质量,适合追求高效准确的开发者。
图像描述的核心目标是将视觉信号无损压缩为文本。然而,依赖人工标注参考文本进行评估,迫使模型模仿特定人类注释风格,掩盖了先进基础模型的真实描述能力。这一系统性偏差引发关键问题:遥感图像描述是否真需任务微调,还是性能差距只是评价标准缺陷所致?为此,我们提出ReconScore,一种无参考评估指标,不依赖文本相似度,而是通过生成文本能否重建原始视觉元素来评估质量,有效消除人为标注偏差。应用该指标后,我们发现一个反直觉结论:未微调的大型多模态模型在真实零样本遥感图像描述任务中优于微调模型。基于此发现,我们提出RemoteDescriber——完全无需训练的生成方法,利用ReconScore作为自校正机制,迭代优化大模型输出语义精度,无额外计算开销。大量实验证明,RemoteDescriber在三个数据集上达到领先性能。同时验证了ReconScore的可靠性,并分析了传统指标的缺陷。代码已公开于https://github.com/hhu-czy/RemoteDescriber。
原文摘要 · Abstract (English)
The core objective of image captioning is to achieve lossless semantic compression from visual signals into textual modalities. However, the reliance on manually curated reference texts for evaluation essentially forces models to mimic specific human annotation styles, thereby masking the true descriptive capabilities of advanced foundation models. This systemic misalignment prompts a critical question: Is task-specific fine-tuning truly necessary for Remote Sensing Image Captioning, or is the perceived performance gap merely an artifact of flawed evaluation criteria? To investigate this discrepancy, we propose ReconScore, a novel reference-free evaluation metric. Rather than computing textual similarities, we assess caption quality by its capability to reconstruct the original visual elements solely from the generated text, effectively neutralizing human annotation biases. Applying this metric, we uncover a profound, counterintuitive truth: inherently powerful, unfine-tuned MLLMs surpass their fine-tuned counterparts in authentic zero-shot RSIC tasks. Driven by this structural discovery, we introduce RemoteDescriber, a completely training-free generation methodology. By employing ReconScore as a self-correction mechanism, we iteratively refine the semantic precision of MLLM outputs without any computational fine-tuning overhead. Comprehensive experiments demonstrate that RemoteDescriber achieves state-of-the-art performance on three datasets. Furthermore, we validate ReconScore's reliability and analyze the flaws of traditional metrics. Our code is available at https://github.com/hhu-czy/RemoteDescriber.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。