用文字描述引导图像复杂度评估,提升模型泛化能力。
Describe-to-Score: A text-guided framework for image complexity assessment
- 训练时融合图文语义,让视觉模型学习复杂度的高层规律
- 在IC9600上达最优,且推理无需额外计算开销
- 适合需要高精度复杂度判断的图像分析任务
准确评估图像复杂度(IC)对众多视觉任务至关重要,但现有方法主要依赖视觉特征,难以捕捉人类判断中的高层语义。本文提出一种多模态视角,将视觉表示与基于图像描述的文本语义相结合,丰富表征空间并提供视觉难推断的结构线索。从信息论和表征角度出发,分析表明语义引导可正则化假设空间,促进更稳定的泛化。我们提出D2S(Describe-to-Score)框架:仅在训练阶段使用文本语义来正则化视觉复杂度建模,推理时保持纯视觉流程,无额外多模态开销。具体通过特征对齐和熵分布对齐,使视觉编码器内化与复杂度相关的语义规律。实验显示,D2S在IC9600基准上达到当前最佳性能,并在无参考图像质量评估(NR-IQA)任务中保持竞争力。进一步分析揭示了小样本训练下的样本不平衡问题及跨数据集迁移中的分布偏移限制。代码已公开于https://github.com/xauat-liushipeng/D2S。
原文摘要 · Abstract (English)
Accurately assessing image complexity (IC) is essential for many vision tasks, yet existing approaches rely almost exclusively on visual features and therefore fail to capture the high-level semantics that humans often use when judging complexity. We introduce a multimodal perspective for IC modeling by integrating visual representations with caption-derived textual semantics. This integration enriches the representational space and provides complementary structural cues that are difficult to infer from vision alone. From an information theoretic and representation viewpoint, we offer an idealized analysis suggesting how semantic guidance can regularize the hypothesis space and support more stable generalization. We propose D2S (Describe-to-Score), a text-guided framework that uses caption-derived semantics only during training to regularize visual complexity modeling, while preserving a vision-only inference pipeline with no additional multimodal overhead at inference. Concretely, D2S transfers semantic structure into the visual branch through feature alignment and entropy distribution alignment, encouraging the visual encoder to internalize complexity-relevant semantic regularities. Experiments show that D2S achieves state-of-the-art performance on the IC9600 benchmark and remains competitive on no-reference image quality assessment (NR-IQA) tasks. Additional analyses further clarify the sample-imbalance issue in the small samples training setting and the distribution-shift limitations observed in cross-dataset transfer. Code is available at: https://github.com/xauat-liushipeng/D2S.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。