解耦专家角色，让大模型在不同领域更精准地切换专业身份
FOCUS: Decoupling Expert Personas in LLMs to Enhance Domain Expert Capabilities
- 通过正交分解分离不同领域的专家特征向量
- 在金融、医疗等任务上准确率提升，跨领域干扰减少
- 适合需要精细角色控制的高风险场景应用
大型语言模型可展现多样化的角色特征，激活专家角色已被证明能提升专业能力与任务准确率。然而，现有角色控制方法常存在跨领域耦合问题，导致在医疗等高谨慎领域行为过于激进，或在金融交易等风险敏感领域表现过度保守。为此，我们提出FOCUS（细调与正交控制解耦角色）。该方法首先自动提取模型中的专家角色向量，再通过正交分解实现领域专属角色的解耦，并引入专家门控模块，根据任务上下文自适应激活对应角色。结合两阶段训练策略与门控选择正则化，模型能够为单领域和跨领域任务学习到合适的角色激活方式。在金融、法律、医疗及跨领域基准测试中，FOCUS显著提升了任务准确率，优于现有角色控制方法。代码已公开于 https://anonymous.4open.science/r/openpersona-48F4。
原文摘要 · Abstract (English)
Large Language Models (LLMs) can exhibit diverse personas, and activating expert personas has been shown to improve domain expertise and task accuracy. However, existing persona control methods often suffer from cross-domain coupling, which may lead to overly aggressive behavior in high-caution domains such as healthcare, or excessive conservatism in risk-sensitive domains such as financial trading. To address this issue, we propose FOCUS (\textbf{\underline{F}}ine-tuning with \textbf{\underline{O}}rthogonal \textbf{\underline{C}}ontrol for \textbf{\underline{U}}ncoupled persona\textbf{\underline{S}}). FOCUS first automatically extracts expert persona vectors from LLMs, then applies orthogonal decomposition to decouple domain-specific expert personas, and finally introduces an expert gating module to adaptively control persona activation according to task contexts. With a two-stage training strategy and a gated selection regularizer, the model learns to activate appropriate personas for both single-domain and cross-domain tasks. Experiments on financial, legal, medical, and cross-domain benchmarks show that FOCUS improves task accuracy and outperforms existing persona control methods. Our code is available at \href{https://anonymous.4open.science/r/openpersona-48F4}{this url}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。