大模型能自动学出完整正确的逻辑程序，省去人工编写
Distilling Answer Set Programming Theories from Large Language Models
- 用大模型在1小时内从零生成答案集编程理论
- 顶级模型在CLEVR上准确率达92.7%-100%，但跨数据集性能波动大
- 适合研究神经符号系统或想快速构建逻辑规则的开发者
从零编写答案集编程（ASP）理论既困难又耗时。本文采用神经符号方法，研究模型是否能在闭环求解器支持下，仅凭单一提示和空文件，在一小时内推导出完整且正确的理论。实验以视觉问答（VQA）为应用领域，选用三个公开且非平凡的数据集（CLEVR、GQA、CLEVRER）。测试了九种不同规模的模型：四款前沿模型（Claude Sonnet 4.6、Claude Opus 4.7、GPT-5、DeepSeek V4 Pro）、两款中等水平模型（DeepSeek V4 Flash、gpt-oss-120b），以及三款开源权重模型（qwen3.6-27b、gpt-oss-20b、qwen3.5-9b）。四款前沿模型在CLEVR上达到100%准确率，在GQA上为92.8%-98.8%；在CLEVRER上，Sonnet、Opus和DeepSeek V4 Pro得分介于92.7%-95.3%。GPT-5在CLEVR上达98.7%，但在GQA下降至41.8%，在CLEVRER为86.7%。引入其他数据集的手写参考理论对前三款前沿模型影响不超过±3.4个百分点，但使GPT-5准确率降低3-19个百分点。代码、提示与生成的理论均已开源。
原文摘要 · Abstract (English)
Writing Answer Set Programming (ASP) theories from scratch is a difficult and time-consuming task. We take a neurosymbolic approach to study whether a model can distill complete and correct theories, given a fixed agent harness with the solver in the loop. The protocol is dataset-agnostic: with a single prompt and an empty file as the starting point the model is given a 1-hour time limit to derive a complete theory. We chose VQA as the application domain, three benchmarks (CLEVR, GQA, CLEVRER), as these are publicly available and non-trivial. In order to study the model scale required for solving this task we nine different models: four frontier (Claude Sonnet 4.6, Claude Opus 4.7, GPT-5, DeepSeek V4 Pro), two mid-tier (DeepSeek V4 Flash, gpt-oss-120b), and three open-weights (qwen3.6-27b, gpt-oss-20b, qwen3.5-9b). Three of four frontier models reach 100% on CLEVR and 92.8%-98.8% on GQA; on CLEVRER, Sonnet, Opus, DeepSeek V4 Pro score 92.7%-95.3%. GPT-5 reaches 98.7% on CLEVR but drops to 41.8% on GQA and to 86.7% on CLEVRER. Adding handwritten reference theories from other datasets moves the other three frontier models by at most +/-3.4 pp but reduces GPT-5's accuracy by 3-19 pp. We release the code, prompts, and theories distilled.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。