arXiv:2608.04175cs.CV2026-08

用单张照片生成热成像和深度图,实现边缘设备上的精准飞机分类。

TriCLE: Tri-Modal Vision-Language Reasoning for Edge-Deployed Fine-Grained Clustering

论文配图:TriCLE: Tri-Modal Vision-Language Reasoning for Edge-Deployed Fine-Grained Clustering
图 1 · 摘自论文原文
  • 从单张RGB图像生成伪热成像与深度图,融合多模态信息进行分类。
  • 在测试集上达到78.00%准确率和0.793加权F1,输出可解析率达94.00%。
  • 模型可压缩至4比特量化,8GB内存下每帧处理仅需1.48秒,适合边缘部署。

用于航空观测的边缘平台需在内存有限、计算能力弱且连接不稳定的条件下解读飞机图像。传统仅依赖RGB的识别模型及通用视觉语言模型在此场景下表现不佳,尤其当真实热成像与激光雷达数据不可用时。本文提出TriCLE,一种面向边缘部署的三模态视觉-语言系统,用于飞机细粒度分类。仅需一张RGB飞机图像,TriCLE即可生成结构保持的类FLIR热成像图与伪LiDAR深度投影,再将对齐后的三模态视图与任务指令输入紧凑的Qwen3-VL骨干网络。模型基于推进方式、机身份组、尺寸、设计年代与构型等工程相关维度对齐专家飞机分类体系,输出反映工程语义相似性而非仅表面外观。我们评估了监督微调、旋转保持微调及三种策略对齐方法:GRPO、GSPO与DAPO。序列级GSPO表现最佳,在验证集上达88.33%准确率与0.91加权F1;在保留测试集上,准确率达78.00%,加权F1为0.793,同时保持94.00%的输出可解析率。经4比特量化与注意力-内存优化后,该40亿参数模型可在8GB设备上部署,每帧三模态输入处理耗时1.48秒。结果表明TriCLE是可解释、可行于边缘的飞机分组原型,但仍需在真实对齐的热成像与激光雷达流上进一步验证。

原文摘要 · Abstract (English)

Edge platforms used for aerial observation must interpret aircraft imagery under limited memory, limited compute, and intermittent connectivity. This setting is difficult for standard RGB-only recognition models and general-purpose vision-language models, especially when calibrated thermal and LiDAR aircraft data are unavailable. We present TriCLE, an application-oriented tri-modal vision-language system for aircraft taxonomic grouping under edge constraints. From a single RGB aircraft image, TriCLE generates a structure-preserving FLIR-style thermal view and a pseudo-LiDAR depth projection, then fuses the aligned views with task instructions in a compact Qwen3-VL backbone. The model is aligned to an expert aircraft taxonomy based on propulsion, airframe family, size, design era, and configuration, so its outputs reflect engineering-relevant similarity rather than only surface appearance. We evaluate supervised fine-tuning, rotation-preserving SFT, and three policy-alignment strategies: GRPO, GSPO, and DAPO. Sequence-level GSPO gives the strongest validation performance, reaching 88.33\% validation accuracy and 0.91 weighted F1 on valid aircraft outputs. On a held-out aircraft test partition, GSPO achieves 78.00\% accuracy and 0.793 weighted F1 while preserving 94.00\% parseable output formatting. After 4-bit quantization and attention-memory optimization, the aligned 4B model fits an 8GB deployment target and processes each tri-modal triplet in 1.48 seconds. These results support TriCLE as a practical prototype for interpretable, edge-feasible aircraft grouping, while emphasizing the need for further validation on real aligned thermal and LiDAR sensor streams.

边缘计算三模态飞机分类量化部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。