让AI自动优化提示词，生成更懂用户的任务解释
Mind the Prompt: Self-adaptive Generation of Task Plan Explanations via LLMs
- 将提示工程建模为认知决策过程，动态捕捉用户状态
- 在两个真实系统中验证，解释质量显著提升
- 适合需要人机协作的复杂系统开发与调试
将大语言模型（LLMs）融入复杂软件系统，可生成对透明化人工智能过程（如自动化任务规划）的人类可理解解释。然而，这些解释的质量和可靠性高度依赖有效的提示工程。由于缺乏对不同利益相关者如何制定和优化提示的系统性理解，自动化该过程的工具难以发展。我们提出COMPASS（COgnitive Modelling for Prompt Automated SynthesiS），一个概念验证性的自适应方法，将提示工程形式化为认知与概率决策过程。COMPASS通过隐变量建模用户潜在认知状态（如注意力、理解度、不确定性）及可观测交互线索，构建部分可观测马尔可夫决策过程（POMDP），其合成策略支持解释与提示的自适应生成。我们在两个多样化的信息物理系统案例研究中评估了COMPASS的自适应解释生成能力及其质量，涵盖定量与定性分析。结果表明，将人类认知与用户画像反馈整合进复杂任务规划系统的自动提示合成是可行的。
原文摘要 · Abstract (English)
Integrating Large Language Models (LLMs) into complex software systems enables the generation of human-understandable explanations of opaque AI processes, such as automated task planning. However, the quality and reliability of these explanations heavily depend on effective prompt engineering. The lack of a systematic understanding of how diverse stakeholder groups formulate and refine prompts hinders the development of tools that can automate this process. We introduce COMPASS (COgnitive Modelling for Prompt Automated SynthesiS), a proof-of-concept self-adaptive approach that formalises prompt engineering as a cognitive and probabilistic decision-making process. COMPASS models unobservable users' latent cognitive states, such as attention and comprehension, uncertainty, and observable interaction cues as a POMDP, whose synthesised policy enables adaptive generation of explanations and prompt refinements. We evaluate COMPASS using two diverse cyber-physical system case studies to assess the adaptive explanation generation and their qualities, both quantitatively and qualitatively. Our results demonstrate the feasibility of COMPASS integrating human cognition and user profile's feedback into automated prompt synthesis in complex task planning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。