表单格式强制语言模型编造答案，即使证据不足也难逃幻觉。
PhantomFill: When the Form Demands an Answer, Language Models Invent One
- 在固定格式下，模型会强制生成虚构内容，哪怕输入无解
- 13个模型中10个在必填字段下幻觉率达100%，40次全编造
- 开源模型无视“证据不足”选项，拒绝使用逃逸机制
生产环境中的语言模型不写散文，而是填表：JSON字段、函数参数、抽取模板。我们发现，表单结构本身会引发幻觉。对同一输入提出相同问题，仅改变输出格式，十三个模型的表现截然不同。输入设计为无法回答：一条有12,400个点赞但无可见回复的病毒帖，一个未录音的支持工单。自由文本中，GPT-5.5 98% 的时间承认无回复数据；但若要求提供情感字段的JSON，同一模型40次全编造。它虚构未见人群的情绪，引用从未听过的客户。在十一个模型中，必填字段使幻觉率达到100%。明确设置“证据不足”选项仅拯救前沿模型——九个开源模型全部无视。在语法约束解码下，五个开源模型在203次试验中零次使用逃逸标记，仅在无代价字段使用12次。它们能输出该词，却拒绝在需付出代价时使用。直接指令‘不要推断情绪’被结构覆盖，六模型中有四失效。规模越大越诚实？非也：同一模型族中，最小模型拒绝，中型编造，最大又拒绝。在格式压力下的诚实，是训练结果，却无人测量。幻觉藏身于无法规避的场合：必选枚举、最小数量数组，这些场景无法附带免责声明。我们发布PhantomFill基准，具备确定性评分与两个可报告指标：强迫幻觉率（Coerced Fabrication Rate）和逃逸利用率（Escape Utilization Rate）。测试的修复只需一行模式定义，而暴露的失败无处不在。
原文摘要 · Abstract (English)
Language models in production do not write prose. They fill forms: JSON fields, function arguments, extraction templates. We show that the form itself causes hallucination. We ask thirteen models the same question about the same input and change only the answer format. The inputs are built so the question cannot be answered: a viral post showing 12,400 likes but no visible replies, a support ticket whose call was never transcribed. In free text, GPT-5.5 says there is no reply data 98% of the time. Given a required JSON field for sentiment, the same model invents an answer 40 times out of 40. It fabricates the mood of crowds it never saw and quotes customers it never heard. Required fields drive fabrication to 100% in ten of thirteen models. An explicit "insufficient evidence" option rescues only the frontier: all nine open-weight models ignore it. Under grammar-constrained decoding, where the escape token is guaranteed reachable by the sampler, five open models spend it zero times out of 203 trials on the three fields that carry the fabrication, and twelve times on the one field where escaping concedes nothing. They can emit the word. They decline to spend it where it costs them an answer. A direct instruction, do not infer sentiment, is overridden by the schema in four of six models. Resistance does not come with scale: within a single model family, the smallest model refuses, the mid-sized model fabricates, the largest refuses again. Honesty under format pressure is a training outcome that no one is measuring. Fabrication hides where hedging is impossible: in required enums and minimum-count arrays, fields where no disclaimer fits. We release PhantomFill, a benchmark with deterministic scoring and two reportable numbers: the Coerced Fabrication Rate and the Escape Utilization Rate. The fix we test is one line of schema. The failure we measure is everywhere.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。