arXiv:2505.21342cs.CL2025-05被引 4

构建1.4万条专利权利要求的明确性标注数据集,助力自动审查系统研发。

PEDANTIC: A Dataset for the Automatic Examination of Definiteness in Patent Claims

  • 用LLM自动从美国专利局文件提取理由,构建标注数据集
  • 1.4万条专利文本经人工验证,标注准确率高
  • 发现大模型虽能识别原因但预测能力不如传统方法

专利权利要求界定发明保护范围,若存在模糊表述将被驳回。在美国,这被称为“明确性不足”(35 U.S.C § 112(b)),是申请被拒的常见原因。自动开展明确性审查可提升撰写与审查效率,但此前缺乏已标注数据集。本文提出PEDANTIC(专利明确性审查语料库),包含1.4万条来自自然语言处理领域专利申请的美国专利权利要求,并标注了明确性缺陷原因。该数据集通过全自动流程构建:从美国专利商标局(USPTO)获取审查意见书,利用大语言模型(LLMs)提取驳回理由,再经人工验证确保质量。为深入评估模型表现,我们引入以大模型为裁判的评测机制,对比各模型提出的理由与审查员原始理由的自由文本推理。结果显示,基于Qwen 2.5 32B和72B的大模型在明确性预测任务中仍无法超越逻辑回归基线,尽管其常能正确识别问题根源。PEDANTIC为专利AI研究提供宝贵资源,代码与数据将公开发布。

原文摘要 · Abstract (English)

Patent claims define the scope of protection for an invention. If there are ambiguities in a claim, it is rejected by the patent office. In the US, this is referred to as indefiniteness (35 U.S.C § 112(b)) and is among the most frequent reasons for patent application rejection. The development of automatic methods for patent definiteness examination has the potential to make patent drafting and examination more efficient, but no annotated dataset has been published to date. We introduce PEDANTIC (Patent Definiteness Examination Corpus), a novel dataset of 14k US patent claims from patent applications relating to Natural Language Processing (NLP), annotated with reasons for indefiniteness. We construct PEDANTIC using a fully automatic pipeline that retrieves office action documents from the USPTO and uses Large Language Models (LLMs) to extract the reasons for indefiniteness. A human validation study confirms the pipeline's accuracy in generating high-quality annotations. To gain insight beyond binary classification metrics, we implement an LLM-as-Judge evaluation that compares the free-form reasoning of every model-cited reason with every examiner-cited reason. We show that LLM agents based on Qwen 2.5 32B and 72B struggle to outperform logistic regression baselines on definiteness prediction, even though they often correctly identify the underlying reasons. PEDANTIC provides a valuable resource for patent AI researchers, enabling the development of advanced examination models. We will publicly release the dataset and code.

专利分析大模型评测数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。