用社交媒体图片测冰雹大小，无需训练就能准到1.12厘米误差。
Detection and Measurement of Hailstones with Multimodal Large Language Models
- 用多模态大模型从社交图片中识别并测量冰雹，分单阶段和双阶段提示。
- 最佳模型平均绝对误差仅1.12厘米，双阶段提示提升结果可靠性。
- 适合气象监测、灾害评估，尤其适合快速获取灾情空间信息的人。
本研究探索利用社交媒体与新闻图像检测并测量冰雹，基于预训练多模态大语言模型。数据集包含奥地利2022年1月至2024年9月间记录的474张冰雹图像，冰雹最大直径为2至11厘米。我们估算冰雹直径，并比较四种模型在单阶段与双阶段提示策略下的表现。后者引入人体手部等参照物提供尺寸线索。结果显示，预训练模型无需微调即可实现平均绝对误差1.12厘米的测量精度；相较于单阶段提示，双阶段提示显著提升多数模型的可靠性。研究表明，这些即插即用模型能补充传统冰雹传感器，从社交媒体图像中提取丰富且空间密集的信息，实现对强天气事件的快速、细致评估。尽管社交媒体图像的自动实时采集仍待解决，但该方法未来可直接应用于冰雹事件监测。
原文摘要 · Abstract (English)
This study examines the use of social media and news images to detect and measure hailstones, utilizing pre-trained multimodal large language models. The dataset for this study comprises 474 crowdsourced images of hailstones from documented hail events in Austria, which occurred between January 2022 and September 2024. These hailstones have maximum diameters ranging from 2 to 11cm. We estimate the hail diameters and compare four different models utilizing one-stage and two-stage prompting strategies. The latter utilizes additional size cues from reference objects, such as human hands, within the image. Our results show that pretrained models already have the potential to measure hailstone diameters from images with an average mean absolute error of 1.12cm for the best model. In comparison to a single-stage prompt, two-stage prompting improves the reliability of most models. Our study suggests that these off-the-shelf models, even without fine-tuning, can complement traditional hail sensors by extracting meaningful and spatially dense information from social media imagery, enabling faster and more detailed assessments of severe weather events. The automated real-time image harvesting from social media and other sources remains an open task, but it will make our approach directly applicable to future hail events.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。