用心理学话术改写提示词，发现越催促越容易出错。
Do Influence Tactics Matter? Investigating Prompt Framing Effects in LLM Code Generation
- 基于心理学策略设计8类提示模板，测试对代码生成的影响。
- 强调紧迫性的提示导致正确率和安全性下降，最大降幅达12%。
- 适合关注AI提示工程与人机协作透明性的开发者参考。
大型语言模型（LLMs）正越来越多地融入软件工程流程，协助开发者编写、调试、测试和维护代码。尽管提示词的措辞和结构已知会影响模型表现，但基于心理学的提示框架影响仍未知。本研究探究人类用于说服或激励他人的心理策略是否可转化为更有效的提示框架，进而影响LLM在编码任务中的行为。基于Yukl & Falbe的经典分类，我们将八种影响力策略（如理性说服、讨好、交换等）转化为可复现的提示模板，并在五个主流开源大模型上使用LiveCodeBench和SWE-bench Verified两个基准进行评估。从功能正确性、质量、可维护性和安全性四个维度评估输出代码。结果表明，强调紧迫性的提示框架与较低的正确率和安全性相关，最大降幅达12%。这是首个关于软件工程任务中影响力提示框架的大规模实证研究，为理解语言线索如何塑造LLM输出提供了洞见。研究最后提出设计透明、可解释的人机交互提示的实践建议。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly integrated into software engineering workflows, helping developers write, debug, test, and maintain code. While prompt wording and structure are known to influence model performance, the impact of psychologically inspired prompt framings remains unexplored. This study investigates whether different psychology-based communication strategies that humans use to persuade or motivate others can lead to more effective prompt framing, which may, in turn, affect LLM behaviour in coding tasks. Drawing on Yukl & Falbe's well-known taxonomy, we operationalized eight influence tactics (like rational persuasion, ingratiation, and exchange) into reproducible prompt templates. These prompt templates were evaluated across five leading open-weight LLMs using two widely adopted benchmarks: LiveCodeBench and SWE-bench Verified. We assessed the resulting code output on four key software quality dimensions: functional correctness, quality, maintainability, and security. Our results show that certain influence-induced prompt framings, particularly those emphasizing urgency, were associated with reduced correctness and security. This work presents the first large-scale empirical study of influence-induced prompt framing in software engineering tasks, offering insights into how linguistic cues may shape LLM outputs. We conclude with practical insights for designing transparent and interpretable human-AI interactions in code generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。