用多模态检索增强生成技术，帮维修人员快速找飞机手册里的图文信息。
Reducing Technician Search Burden: A Multimodal RAG for Cessna 172 Maintenance Manual
- 结合文本与图像的多模态检索，精准定位手册中的图文内容。
- 检索召回率93.37%，生成答案与标准答案语义相似度达87.20%。
- 速度快、成本低，适合实际维修场景使用，且结果可解释。
正确使用飞机维护手册对确保维修质量至关重要，其包含操作流程、图示、警告和规范信息。然而，维修人员常因手册难导航、耗时长而回避查阅。现有基于检索增强生成（RAG）的模型仅支持文本检索。本研究针对通用航空中广泛使用的塞斯纳172型飞机维护手册（C172-MM），开发了多模态手册检索器（MMR），可检索包含图文的页面。通过合成查询评估检索性能，覆盖流程、图示、警告/安全信息及规范内容，实现93.37%的recall@5。进一步构建多模态RAG（MRAG）管道，将检索结果输入视觉语言模型生成回答，与真实答案的语义相似度达87.20%。还评估了三项实际可行性：平均检索五页耗时11.93秒，生成响应需4.95秒，每查询成本0.0091美元，通过热力图可视化验证了结果可解释性。结果表明，该MRAG系统能显著减少技术人员查找手册和获取多模态信息的时间。
原文摘要 · Abstract (English)
Proper use of the aircraft maintenance manual is essential for correct maintenance, providing procedures, diagrams, cautions, and specifications. However, technicians often avoid consulting it because it is difficult to navigate and time-consuming under strict schedules. Retrieval augmented generation (RAG) models have recently been introduced in aircraft maintenance, yet existing models focus solely on textual retrieval. This research therefore targeted the Cessna 172 Maintenance Manual (C172-MM), widely used in general aviation, and developed a multimodal manual retriever (MMR) capable of retrieving multimodal manual pages. Retrieval performance was evaluated using synthetic queries covering procedures, diagrams, caution/safety information, and specifications; the MMR achieved 93.37% recall@5. Beyond retrieval, a multimodal RAG (MRAG) pipeline was examined, in which retrieved pages were input to a vision-language model that generated responses to the synthetic queries, achieving 87.20% semantic similarity to ground-truth answers. Three practical feasibilities were also assessed: inference time, operational cost, and interpretability. Average retrieval time for five pages was 11.93 seconds and response generation took 4.95 seconds, at $0.0091 per query, while interpretability was validated through heatmap visualizations. These results indicate that the MRAG pipeline for the C172-MM can reduce the time technicians spend searching manuals and retrieving multimodal information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。