用微调的本地大模型实现高精度安全文档分类，兼顾隐私与准确性。
Security Document Classification with a Fine-Tuned Local Large Language Model: Benchmark Data and an Open-Source System
- 基于微调的Qwen 3.5 27B模型构建本地分类系统
- 主测试集上达到95.0%类别准确率，远超商用模型
- 开源可部署，适合需数据本地化的安全场景
组织在扫描敏感信息文档时面临现实困境：云服务需将数据外传，而规则工具常因依赖上下文而漏判威胁。本文提出TorchSight，一个基于微调的Qwen 3.5 27B模型的开源本地安全文档分类系统。模型在13个宽松许可来源的78,358个样本及GPT-4生成数据（覆盖7类51子类）上训练。主评估在1,000份文档上取得95.0%类别准确率（95%置信区间：93.5–96.2），相同提示协议下商用模型表现仅为75.4–79.9%。在500份独立外部样本上达93.8%准确率，表明性能可泛化，但受数据构成和边界案例影响。结果表明，微调的本地模型可在保障数据控制权的同时实现高精度分类。
原文摘要 · Abstract (English)
Organizations that scan documents for sensitive information face a practical problem. Cloud services require data to be sent to external infrastructure, while rule-based tools often miss threats that depend on context. This study presents TorchSight, an open-source local system for security document classification built around a fine-tuned Qwen 3.5 27B model. The model was trained on 78,358 samples from 13 permissively licensed sources and GPT-4 synthetic data covering seven security categories and 51 subcategories. In the main evaluation on 1,000 documents, the model reached 95.0% category-level accuracy (95% confidence interval: 93.5-96.2). The tested commercial models scored 75.4-79.9% under the same prompting protocol. On a separate external set of 500 held-out samples, the model reached 93.8% accuracy, which suggests that performance extends beyond the main benchmark, although the margin depends on dataset composition and difficult boundary cases. The results show that a fine-tuned local model can support accurate security document classification while keeping document processing under local control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。