通过主成分分析实现大模型讨好行为的精准双向控制
PCA-guided Activation Scaling for Monotonic Bidirectional Control over LLM Sycophancy
- 用主成分分析分解激活值，分离出讨好与诚实维度
- 不同层施加不对称缩放，实现可预测的渐变控制
- 在三模型三数据集上提升控制精度，适合需调节态度的场景
大语言模型存在讨好倾向，即无论事实真假都迎合用户观点，可能加剧误解。但完全消除又可能矫枉过正。有效控制需能同时减弱和增强这种倾向，且效果可预测、渐进。现有方法无法保证在不同模型和数据集上，操控强度与行为变化之间保持双向、单调关系。本文提出PCA引导的激活缩放（PAS）框架：将残差流激活分解为由主成分识别的讨好-诚实子空间及正交余量，对两者分别施加不同缩放指数，实现单调、双向控制。在三个大模型和三个数据集上，PAS达成强单调性（斯皮尔曼相关系数ρ = +0.92），平均每方向调整15.4%，优于基线的8.7%。消融实验证明，分解机制、非对称指数和层选择均对维持单调控制至关重要。数据与代码已开源。
原文摘要 · Abstract (English)
Large language models (LLMs) exhibit sycophancy, a tendency to agree with user beliefs regardless of factual accuracy. This can reinforce misconceptions, but eliminating it entirely risks over-correction against valid opinions. Effective control must therefore both reduce and increase sycophancy with predictable and gradual effect. Yet, existing methods fail to ensure a bidirectional and monotonic relationship between steering strength and behavioral outcome across models and datasets. We introduce PCA-guided Activation Scaling (PAS), an activation steering framework that decomposes residual stream activations into a PCA-identified sycophancy-honesty subspace and an orthogonal residual, then applies distinct scaling exponents to achieve monotonic, bidirectional control. Across three LLMs and three datasets, PAS achieves strong monotonicity (Spearman $ρ$ = +0.92) and an average shift of 15.4% per direction, compared with 8.7% for the baselines. Ablation studies confirm that the decomposition, asymmetric exponents, and layer selection are each essential for maintaining monotonic control. The data and code are available at https://github.com/Bellafc/PCS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。