首个专用于专利描述的多维评估框架，兼顾技术与法律合规性。
Pat-DEVAL: Chain-of-Legal-Thought Evaluation for Patent Description
- 引入法律思维链机制，按专利法要求逐层分析文本
- 在专家评测数据集上相关性达0.69，法律合规项达0.73
- 适合自动化专利撰写系统开发者和知识产权从业者
专利描述需在提供完整技术披露的同时满足可实施性、书面描述等严格法律要求。尽管大语言模型已实现端到端自动撰写专利，但现有评估方法无法有效衡量长文本的结构连贯性及专利法合规性。本文提出Pat-DEVAL，首个专用于专利描述内容的多维度评估框架。基于‘大模型作为裁判’范式，引入法律思维链（CoLT）机制，强制执行专利法特有的顺序化分析。在由专利专家标注的Pap2Pat-EvalGold数据集上实验表明，Pat-DEVAL达到0.69的皮尔逊相关系数，显著优于基线指标和现有大模型评估器。尤其在法律专业合规性方面相关性达0.73，证明显式注入法定约束对捕捉法律有效性至关重要。该框架为确保技术严谨性与法律合规性的统一提供了方法论基础，推动自动化专利撰写系统的实际应用。
原文摘要 · Abstract (English)
Patent descriptions must deliver comprehensive technical disclosure while meeting strict legal standards such as enablement and written description requirements. Although large language models have enabled end-to-end automated patent drafting, existing evaluation approaches fail to assess long-form structural coherence and statutory compliance specific to descriptions. We propose Pat-DEVAL, the first multi-dimensional evaluation framework dedicated to patent description bodies. Leveraging the LLM-as-a-judge paradigm, Pat-DEVAL introduces Chain-of-Legal-Thought (CoLT), a legally-constrained reasoning mechanism that enforces sequential patent-law-specific analysis. Experiments validated by patent expert on our Pap2Pat-EvalGold dataset demonstrate that Pat-DEVAL achieves a Pearson correlation of 0.69, significantly outperforming baseline metrics and existing LLM evaluators. Notably, the framework exhibits a superior correlation of 0.73 in Legal-Professional Compliance, proving that the explicit injection of statutory constraints is essential for capturing nuanced legal validity. By establishing a new standard for ensuring both technical soundness and legal compliance, Pat-DEVAL provides a robust methodological foundation for the practical deployment of automated patent drafting systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。