用视觉风格检测天气,轻量高效且实时运行。
Heuristic Style Transfer for Real-Time, Efficient Weather Attribute Detection

- 基于视觉风格设计轻量多任务模型,融合注意力机制
- 在内部测试上F1超96%,零样本跨数据集仍超78%
- 适合嵌入式系统,模块化设计支持灵活扩展
我们提出轻量高效的架构,从RGB图像中检测天气状况,识别天气类型(晴、雨、雪、雾)及11个互补属性(如强度、能见度、地面状态),共53类。研究探讨了天气如何以视觉风格变化体现。采用风格启发技术,包括格拉姆矩阵、截断的ResNet-50(低层与中间层)、以及PatchGAN式结构,在带注意力机制的多任务框架中实现。提出两类模型:RTM(ResNet50-Truncated-MultiTasks)和PMG(PatchGAN-MultiTasks-Gram)及其变体。贡献包括自动化格拉姆矩阵计算、将PatchGAN融入有监督多任务学习、通过局部格拉姆捕捉局部风格以增强空间一致性。同时发布包含503,875张图像、标注12个天气属性的数据集,采用CC-BY许可。模型在内部测试集上F1分数超过96%,在多个外部数据集零样本评估中达78%以上,验证泛化能力。PMG架构参数少于500万,可实时运行且内存占用小,适用于嵌入式系统。模型模块化设计允许按需增删风格或天气相关任务。
原文摘要 · Abstract (English)
We present lightweight and efficient architectures to detect weather conditions from RGB images, predicting the weather type (sunny, rain, snow, fog) and 11 complementary attributes such as intensity, visibility, and ground condition, for a total of 53 classes across the tasks. This work examines to what extent weather conditions manifest as variations in visual style. We investigate style-inspired techniques, including Gram matrices, a truncated ResNet-50 targeting lower and intermediate layers, and PatchGAN-style architectures, within a multi-task framework with attention mechanisms. Two families are introduced: RTM (ResNet50-Truncated-MultiTasks) and PMG (PatchGAN-MultiTasks-Gram), together with their variants. Our contributions include automation of Gram-matrix computation, integration of PatchGAN into supervised multi-task learning, and local style capture through local Gram for improved spatial coherence. We also release a dataset of 503,875 images annotated with 12 weather attributes under a Creative Commons Attribution (CC-BY) license. The models achieve F1 scores above 96 percent on our internal test set and above 78 percent in zero-shot evaluation on several external datasets, confirming their generalization ability. The PMG architecture, with fewer than 5 million parameters, runs in real time with a small memory footprint, making it suitable for embedded systems. The modular design of the models also allows style-related or weather-related tasks to be added or removed as needed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。