用外部知识生成更精准的遥感图像问题，提升问答系统实用性。
Knowledge-aware Visual Question Generation for Remote Sensing Images
- 引入外部知识三元组增强问题上下文理解
- 在两个新标注数据集上优于现有方法
- 适合遥感智能问答与视觉对话研究者
随着遥感图像档案的快速增长，基于图像提问成为获取特定信息或执行图像检索的有效方式。然而，自动生成的问题往往过于简单且依赖模板，阻碍了问答或视觉对话系统的实际应用。为丰富并多样化问题生成，我们提出一种知识感知的遥感视觉问题生成模型 KRSVQG，该模型结合与图像内容相关的外部知识，以提升生成问题的质量和上下文理解能力。模型以图像和来自外部知识源的知识三元组为输入，利用图像描述作为中间表示，增强问题对图像的语义定位。为评估 KRSVQG 性能，我们手动标注了两个数据集：NWPU-300 和 TextRS-300。实验结果表明，KRSVQG 在这两个数据集上均优于现有方法，生成的问题不仅更具知识丰富性，且同时基于图像内容与领域知识进行定位。
原文摘要 · Abstract (English)
With the rapid development of remote sensing image archives, asking questions about images has become an effective way of gathering specific information or performing image retrieval. However, automatically generated image-based questions tend to be simplistic and template-based, which hinders the real deployment of question answering or visual dialogue systems. To enrich and diversify the questions, we propose a knowledge-aware remote sensing visual question generation model, KRSVQG, that incorporates external knowledge related to the image content to improve the quality and contextual understanding of the generated questions. The model takes an image and a related knowledge triplet from external knowledge sources as inputs and leverages image captioning as an intermediary representation to enhance the image grounding of the generated questions. To assess the performance of KRSVQG, we utilized two datasets that we manually annotated: NWPU-300 and TextRS-300. Results on these two datasets demonstrate that KRSVQG outperforms existing methods and leads to knowledge-enriched questions, grounded in both image and domain knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。