arXiv:2607.03320cs.CV2026-07

轻量级双流变压器实现遥感图像变化实时描述生成

LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning

论文配图:LBTCap: A Lightweight Bilateral Transformer for Real-Time Remote Sensing Image Change Captioning
图 1 · 摘自论文原文
  • 采用共享查询键矩阵的双流注意力机制,显式建模变化前后图像特征
  • 仅用3999万参数,变化感知编码器仅278万,推理速度显著提升
  • 在低资源场景下优势明显,适合实际应用中的高效部署

遥感图像变化描述(RSICC)为成对遥感图像的语义变化生成自然语言描述,支持城市规划、灾害响应和环境监测等应用。尽管现有方法具备较强描述准确率,但多数忽视计算效率与推理速度,难以满足实际中的实时需求。为此,本文提出LBTCap,一种基于双流Transformer的轻量级框架,通过联合建模变化前后的图像特征,实现高效处理。具体地,设计双流注意力机制:两时相图像由同一组查询与键矩阵投影生成查询与键,值由二者拼接构成,两个注意力图通过可学习的结构化双流加权融合,而非固定减法。该设计保持双分支显式性的同时保持紧凑性,并结合截断主干网络与分组查询注意力,使整体仅需3999万参数,其中变化感知编码器仅占278万。在两个公开的RSICC数据集上的实验表明,LBTCap在保持或接近顶尖方法准确率的同时,参数量大幅减少,推理速度显著提高;双流结构在低资源条件下优势最为突出,展现出优异的精度-效率权衡,适用于实际遥感变化分析任务。

原文摘要 · Abstract (English)

Remote sensing image change captioning (RSICC) generates natural-language descriptions of semantic changes between paired remote sensing images (RSIs), supporting applications such as urban planning, disaster response, and environmental monitoring. Although recent methods achieve strong captioning accuracy, most overlook computational efficiency and inference speed, which are essential for real-time deployment in practice. To this end, we propose LBTCap, a lightweight RSICC framework built on a bilateral Transformer that jointly models pre- and post-change features for efficient processing of paired RSIs. Specifically, we introduce a bilateral attention mechanism for paired inputs: the two temporal images are projected into separate queries and keys by the same query and key matrices shared across the two images, the value is formed from their concatenation, and the two resulting attention maps are combined by a learnable, structurally bilateral weighting instead of a fixed subtraction. This design keeps both temporal branches explicit while remaining compact, and, together with a truncated backbone and grouped-query attention, LBTCap uses only 39.99M parameters, of which the change-aware encoder accounts for just 2.78M. Extensive experiments on two public RSICC datasets show that LBTCap matches or closely approaches the accuracy of state-of-the-art methods while using far fewer parameters and running at markedly higher inference speed, with the benefit of the bilateral formulation most pronounced in the low-resource setting, demonstrating a favorable accuracy-efficiency trade-off for practical RSICC.

遥感图像变化检测轻量模型实时生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。