用确定性工具替代语言模型生成代码,确保科学分析结果可复现。
It's not the Language Model, it's the Tool: Deterministic Mediation for Scientific Workflows

- 让模型调用预定义的确定性工具,而非生成分析代码。
- 相同输入下,工具输出完全一致,商业模型则结果不一或失败。
- 适合对可复现性要求高的科研场景,尤其处理专有格式数据时。
语言模型虽能生成看似可信的科学分析,但重复运行相同任务却无法保证一致结果:同一查询可能得到不同拟合、峰值位置或分析流程,难以判断哪个输出可信。本文提出类型化中介(typed mediation)模式,即让模型调度确定性工具而非生成分析代码。每个工具封装了一位研究人员针对特定仪器的精确操作流程,通过结构化访谈迁移实现。模型决定调用哪个工具及参数。结果由工具产生,再生不会改变。我们在四个平台(含三个商用基础模型)上对光致发光分析进行四次重复测试,相同提示下,类型化工具输出完全一致;而商用模型在数值和方法上均出现变异,或无法完成任务。该模式已在两个仪器上部署约六个月,用户反馈良好。两项任务极具挑战性:涉及专有二进制格式与按席位授权的软件,迫使工具必须与数据和仪器同处本地环境。我们主张部署拓扑不仅是偏好,而是科学工具中介的结构性需求。该模式使分析时间从数周缩短至分钟级,同时确保跨运行结果一致,满足科学工作的可复现性要求。
原文摘要 · Abstract (English)
Language models can produce convincing scientific analyses, but repeated generations on the same data do not guarantee the same result. A researcher may regenerate an identical query and receive a different fit, a different peak position or a different analysis procedure, without an obvious way to decide which output to trust. We propose typed mediation, a pattern in which the model orchestrates deterministic tools rather than generating analytical code. Each tool encodes one researcher's exact procedure for one instrument, ported through structured interviews. The model selects which tool to call and with what parameters. The tool produces the result. Regeneration does not change it. We evaluate this claim by running the same photoluminescence analysis on four platforms, including three commercial foundation models, four times each with the same prompt. The typed tool produces identical results across all runs. The commercial platforms either vary in numerical output and analytical methodology across runs, or fail to produce valid results on the task. We deploy this pattern on two instruments serving users over approximately six months, with very positive user feedback. Both cases are very challenging: they involve proprietary binary formats and per-seat licensed software, which force the tool to remain on local infrastructure alongside the data and the instrument it operates. We argue that deployment topology is not just a preference, but a structural requirement of scientific tool mediation. The result is a practical pattern for deploying language models in scientific workflows where reproducibility is mandatory, reducing analysis time from weeks to minutes while guaranteeing identical outputs across runs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。