用AI自动生成标注,人工修正少90%,支持多种格式导出。
VisioFirm: Cross-Platform AI-assisted Annotation Tool for Computer Vision
- 结合CLIP与预训练检测器,零样本生成初始标注。
- 实测减少90%人工标注工作量,准确率高且支持交互修改。
- 浏览器端实时分割,离线可用,适合大规模数据标注场景。
AI模型依赖标注数据学习模式并进行预测。传统标注工具需大量手动操作,难以扩展至大规模数据集。为此,我们提出VisioFirm,一款开源Web标注工具,通过AI辅助自动化提升图像标注效率。该工具集成CLIP与Ultralytics等预训练检测器,结合Grounding DINO实现零样本标注,采用低置信度阈值生成初始标注以最大化召回率。在COCO类任务上测试表明,初始预测基本正确,用户可通过支持边界框、方向框和多边形的交互工具进行修正。此外,基于Segment Anything模型,利用WebGPU加速实现浏览器端实时分割。支持YOLO、COCO、Pascal VOC、CSV等多种导出格式,模型缓存后可离线运行。基准测试显示,相比传统方法,人工工作量降低高达90%,同时通过基于CLIP的组件聚类与IoU图冗余抑制机制保障标注准确性。项目开源地址:https://github.com/OschAI/VisioFirm。
原文摘要 · Abstract (English)
AI models rely on annotated data to learn pattern and perform prediction. Annotation is usually a labor-intensive step that require associating labels ranging from a simple classification label to more complex tasks such as object detection, oriented bounding box estimation, and instance segmentation. Traditional tools often require extensive manual input, limiting scalability for large datasets. To address this, we introduce VisioFirm, an open-source web application designed to streamline image labeling through AI-assisted automation. VisioFirm integrates state-of-the-art foundation models into an interface with a filtering pipeline to reduce human-in-the-loop efforts. This hybrid approach employs CLIP combined with pre-trained detectors like Ultralytics models for common classes and zero-shot models such as Grounding DINO for custom labels, generating initial annotations with low-confidence thresholding to maximize recall. Through this framework, when tested on COCO-type of classes, initial prediction have been proven to be mostly correct though the users can refine these via interactive tools supporting bounding boxes, oriented bounding boxes, and polygons. Additionally, VisioFirm has on-the-fly segmentation powered by Segment Anything accelerated through WebGPU for browser-side efficiency. The tool supports multiple export formats (YOLO, COCO, Pascal VOC, CSV) and operates offline after model caching, enhancing accessibility. VisioFirm demonstrates up to 90\% reduction in manual effort through benchmarks on diverse datasets, while maintaining high annotation accuracy via clustering of connected CLIP-based disambiguate components and IoU-graph for redundant detection suppression. VisioFirm can be accessed from \href{https://github.com/OschAI/VisioFirm}{https://github.com/OschAI/VisioFirm}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。