用多维度大模型自动检测专利文本质量并提改进建议
Towards Automated Quality Assurance of Patent Specifications: A Multi-Dimensional LLM Framework
- 分三模块检测合规性、技术连贯性与图注一致性
- 准确率最高达99.74%，发现AI专利存在结构缺陷
- 适合专利审查员和AI辅助写作工具开发者使用
尽管AI drafting 工具在专利撰写中日益重要，但对AI生成专利内容的质量系统评估仍是一个研究空白。为此，我们提出一个框架，通过监管合规性、技术连贯性以及图注一致性检测模块评估专利，并由整合模块生成改进建议。该框架在包含80份人工撰写和80份AI生成专利的综合数据集上进行验证，分别针对10,841个句子、8,924个非模板句子和554张专利图进行评估，三模块平衡准确率分别为99.74%、82.12%和91.2%，均基于专家标注。进一步分析揭示了缺陷在专利章节、技术领域及来源间的分布特征：图-文一致性与技术细节精确度需重点关注；机械工程与建筑领域因技术文档要求复杂，更易出现权利要求与说明书不一致；相较于人工撰写，AI生成专利存在显著结构性缺陷，如图注对齐与交叉引用问题，而人工专利多为表面错误如错别字。
原文摘要 · Abstract (English)
Although AI drafting tools have gained prominence in patent writing, the systematic evaluation of AI-generated patent content quality represents a significant research gap. To address this gap, We propose to evaluate patents using regulatory compliance, technical coherence, and figure-reference consistency detection modules, and then generate improvement suggestions via an integration module. The framework is validated on a comprehensive dataset comprising 80 human-authored and 80 AI-generated patents from two patent drafting tools. Evaluation is performed on 10,841 total sentences, 8,924 non-template sentences, and 554 patent figures for the three detection modules respectively, achieving balanced accuracies of 99.74%, 82.12%, and 91.2% against expert annotations. Additional analysis was conducted to examine defect distributions across patent sections, technical domains, and authoring sources. Section-based analysis indicates that figure-text consistency and technical detail precision require particular attention. Mechanical Engineering and Construction show more claim-specification inconsistencies due to complex technical documentation requirements. AI-generated patents show a significant gap compared to human-authored ones. While human-authored patents primarily contain surface-level errors like typos, AI-generated patents exhibit more structural defects in figure-text alignment and cross-references.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。