让大模型在矛盾问题中生成多套合理解法,突破传统推理局限。
FlowEdit: Information-Theoretic Control of LLM Reasoning Flows for Ill-posed Problems Involving Conflicts

- 用信息论调控模型内部推理路径,实现多分支并行思考。
- 在冲突问题上提升准确率68%,响应信息量增加24%。
- 适合需要应对复杂矛盾的智能系统研发者参考。
大型语言模型(LLMs)在定义明确的推理任务中表现优异,但在开放世界中常因条件不一致、陈述冲突或要求互斥而变得病态,无法产生有效回应。我们提出FlowEdit框架,利用信息论原理量化并调节模型内部推理流,以在单次推理中生成基于不同有效假设的一组替代性回答。该方法通过双目标信息论机制,在模型内部表示层面实现分支感知的推理:最大化各假设到分支结果的信息流,同时最小化兄弟分支间的重叠与条件依赖,从而获得覆盖广、信息丰富的多样化响应。理论证明在边界嵌入ε-充分条件下,可通过可计算的变分界优化底层条件互信息。大量实验表明,FlowEdit优于领先商用模型,在精确集匹配准确率上提升68%,整体响应信息量提升24%。进一步分析显示,流调控在词元流中体现为熵分布的变化:每个分支内熵集中,流边界处熵放大,且随所需分支数增加而增长。
原文摘要 · Abstract (English)
Large Language Models (LLMs) perform strongly on well-specified reasoning tasks with a feasible answer. However, problems encountered in the open world can become ill-posed due to inconsistent conditions, conflicting statements, or mutually incompatible requirements, admitting no valid responses. We argue that reasoning of such ill-posed problems involving conflicts require novel LLM capabilities to make hidden conflicts explicit, maintain competing hypotheses via multiple reasoning branches, and generate alternative responses in a single pass, all of which are challenging due to the limitation of the next-token prediction mechanism in LLMs. To this end, we propose FlowEdit, a novel framework that leverages information-theoretic principles to quantify and regulate internal reasoning flows of LLMs, for generating a full set of alternative responses under valid hypotheses. FlowEdit can be viewed as enforcing a branch-aware reasoning process using two dual information-theoretic objectives on the model's internal reasoning representations: maximizing the information flow from each selected hypothesis to the branch outcome, while minimizing the overlap and conditional dependence across sibling branches, to provide a diverse, informative set of responses with broad coverage. We show that this is achieved through tractable variational bounds under boundary embeddings being ε-sufficient, optimizing the underlying conditional mutual information in LLM reasoning process. Extensive experiments demonstrate that FlowEdit outperforms leading proprietary models, improving exact-set-match accuracy by 68%, while boosting overall response informativeness by 24%. We further show that flow regulation surfaces in the token stream as a redistribution of next-token entropy that concentrates inside each branch, amplifies at flow boundaries, and scales with the number of flows the problem requires.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。