大模型生成选择题时会隐式偏爱特定选项位置，可通过内部信号干预控制。
Do Large Language Models Plan Answer Positions? Position Bias in Multiple-Choice Question Generation
- 分析10个大模型在3项任务中的答案位置分布，发现系统性偏倚
- 隐藏层特征编码正确答案位置的预测信号，表明存在隐式规划
- 用激活操控可部分调控位置偏好，适合需可控生成的研究者
大型语言模型（LLMs）被广泛用于生成多选题（MCQs），理想情况下正确答案应均匀分布于各选项。然而，我们观察到这些模型在生成过程中表现出系统性的位置偏倚。通过对10个LLMs和5个视觉-语言模型（VLMs）在三个MCQ生成任务上的广泛实验，我们发现这种偏倚具有结构性，且在同一模型家族中呈现相似模式。为探究其内在机制，我们进行了探测实验，发现问题题干的隐藏表示中编码了正确答案位置的预测信号，表明答案位置可能在生成过程中被隐式规划。基于此洞察，我们应用激活操控来干预内部表示并影响答案位置。结果表明，操控可部分控制位置偏好，并显著改变答案位置分布。研究为分析大模型中隐式位置规划提供了实用框架，并强调了可控生成对可靠题库构建与评估的重要性。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used to generate multiple-choice questions (MCQs), where correct answers should ideally be uniformly distributed across options. However, we observe that LLMs exhibit systematic position biases during generation. Through extensive experiments with 10 LLMs and 5 vision-language models (VLMs) on three MCQ generation tasks, we show that these biases are structured, with similar patterns emerging within model families. To investigate the underlying mechanisms, we conduct probing experiments and find that hidden representations in the question stem encode predictive signals of the correct answer position, suggesting that answer position may be implicitly planned during generation. Building on this insight, we apply activation steering to manipulate internal representations and influence answer position. Our results show that steering can partially control positional preferences and substantially shift answer position distributions. Our findings provide a practical framework for studying implicit positional planning in LLMs and highlight the importance of controllable generation for reliable MCQ construction and evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。