本地化多模型流程，让开源视觉语言模型自动生成乳腺钼靶报告。
MammoWise: Multi-Model Local RAG Pipeline for Mammography Report Generation
- 基于Ollama的多模型管道，支持零样本、少样本及思维链提示生成报告。
- 引入向量数据库RAG增强上下文，报告质量与分类准确率显著提升。
- 可参数高效微调，兼顾隐私安全与可复现性，适合医院本地部署。
乳腺钼靶筛查量大、时间紧、文档繁重，放射科医生需将细微影像发现转化为一致的BI-RADS评估、乳腺密度分类和结构化报告。尽管近期视觉语言模型（VLMs）实现图像转文本报告，但多数依赖封闭云系统或紧耦合架构，限制了隐私保护、可复现性和可扩展性。我们提出MammoWise，一个本地化多模型流水线，将开源VLMs转化为钼靶报告生成器与多任务分类器。MammoWise支持任意Ollama托管的VLM和钼靶数据集，提供零样本、少样本及思维链提示，并可选通过向量数据库实现病例特异性检索增强生成（RAG）。在VinDr-Mammo和DMID数据集上评估MedGemma、LLaVA-Med和Qwen2.5-VL，涵盖报告质量（BERTScore、ROUGE-L）、BI-RADS分类、乳腺密度与关键发现识别。报告生成表现稳健，且随少样本提示与RAG使用持续提升；分类可行但对模型与数据集选择敏感。对MedGemma进行参数高效微调（QLoRA）后，获得BI-RADS准确率0.7545、密度准确率0.8840、钙化检测准确率0.9341，同时保持报告质量。MammoWise为在统一、可复现的工作流中部署本地VLM于乳腺钼靶报告提供了实用且可扩展的框架。
原文摘要 · Abstract (English)
Screening mammography is high volume, time sensitive, and documentation heavy. Radiologists must translate subtle visual findings into consistent BI-RADS assessments, breast density categories, and structured narrative reports. While recent Vision Language Models (VLMs) enable image-to-text reporting, many rely on closed cloud systems or tightly coupled architectures that limit privacy, reproducibility, and adaptability. We present MammoWise, a local multi-model pipeline that transforms open source VLMs into mammogram report generators and multi-task classifiers. MammoWise supports any Ollama-hosted VLM and mammography dataset, and enables zero-shot, few-shot, and Chain-of-Thought prompting, with optional multimodal Retrieval Augmented Generation (RAG) using a vector database for case-specific context. We evaluate MedGemma, LLaVA-Med, and Qwen2.5-VL on VinDr-Mammo and DMID datasets, assessing report quality (BERTScore, ROUGE-L), BI-RADS classification, breast density, and key findings. Report generation is consistently strong and improves with few-shot prompting and RAG. Classification is feasible but sensitive to model and dataset choice. Parameter-efficient fine-tuning (QLoRA) of MedGemma improves reliability, achieving BI-RADS accuracy of 0.7545, density accuracy of 0.8840, and calcification accuracy of 0.9341 while preserving report quality. MammoWise provides a practical and extensible framework for deploying local VLMs for mammography reporting within a unified and reproducible workflow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。