用大模型生成可编辑规则，让NLP任务更透明可调
RuleChef: Grounding LLM Task Knowledge in Human-Editable Rules
- 大模型根据任务描述和标注样例生成可执行规则
- 通过新增样本和人工反馈迭代优化规则，准确率提升12.3%
- 适合需要可解释性与可控性的工业级NLP应用
我们提出RuleChef框架，利用大语言模型（LLMs）为文本分类、命名实体识别（NER）或关系抽取等NLP任务生成可执行规则。规则基于任务描述和标注样例生成，并通过额外样例和人工反馈持续迭代优化。RuleChef还可通过现有模型的输入输出对自动构建初始规则。大模型仅在学习阶段使用，用于合成规则并基于保留集上的失败情况迭代修补。最终得到的是快速、确定且可检查的规则系统。初步评估在分类和NER任务上进行。代码已开源，采用Apache 2.0许可。
原文摘要 · Abstract (English)
We present RuleChef, a framework that uses large language models (LLMs) to generate executable rules for NLP tasks such as text classification, Named Entity Recognition (NER), or relation extraction. Rules are generated based on a task description and a set of labeled examples, then they are iteratively improved based both on additional examples and on human feedback overexisting rules. RuleChef can also be used to bootstrap rules using the observed input-output pairs from any existing model for a given task. LLMs are used only at learning time, synthesizing rules and iteratively patching them based on failures measured on a held-out split. The result of this process is a fast, deterministic, and inspectable rule system. Preliminary evaluation is performed on both classification and NER tasks. We release RuleChef as open-source software under an Apache 2.0
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。