Mind the Prompt: Self-adaptive Generation of Task Plan Explanations via LLMs

1. Problem Statement

The integration of Large Language Models (LLMs) into complex software systems, particularly for automated task planning, offers significant potential for automation. However, the resulting plans are often opaque to human stakeholders. While LLMs can generate explanations for these plans, the quality and reliability of these explanations depend heavily on prompt engineering.

Current approaches rely on static, generic prompts or ad-hoc human intuition, failing to account for:

• Diverse Stakeholder Needs: Different users (e.g., AI experts, finance managers, non-experts) require explanations with varying granularity, tone, and structure.
• Dynamic Cognitive States: A user's attention, comprehension, and cognitive load fluctuate based on context (e.g., fatigue, time of day).
• Lack of Systematic Adaptation: There is no principled framework to automatically refine prompts based on user feedback and latent cognitive states without imposing a heavy cognitive burden on the user.

The core challenge is to create a system that self-adapts the generation of task plan explanations to align with a user's profile, real-time cognitive state, and feedback history.

2. Methodology: COMPASS

The authors propose COMPASS (COgnitive Modelling for Prompt Automated SynthesiS), a proof-of-concept self-adaptive framework. It treats prompt engineering as a cognitive and probabilistic decision-making process modeled using a Partially Observable Markov Decision Process (POMDP).

System Architecture

COMPASS operates within a MAPE-K (Monitor, Analyze, Plan, Execute, Knowledge) loop and consists of two main stages:

Stage S1: Problem Generation

• Input: Natural language descriptions of a planning problem (e.g., from stakeholders).
• Process: A GenAI-based Planning Problem Generator converts the natural language into a structured, machine-readable format (e.g., JSON or PDDL) required by an off-the-shelf AI planner.
• Output: Feasible task plans generated by an AI planner (e.g., Fast Downward, NSPH).

Stage S2: Adaptive Synthesis of Explanations
This stage dynamically generates human-readable explanations tailored to the user.

1. Monitoring: The system monitors user profiles, feedback (accept/reject), and cognitive skill data (predicted attention/understanding levels via Deep Learning models).
2. Profile POMDP Modeling:
   • Hidden States: The user's true cognitive state (e.g., actual attention level) is unobservable.
   • Observations: The system observes noisy proxies, such as predicted cognitive levels from a DNN and explicit user feedback (accept/reject).
   • Actions: Selecting specific prompt parameters (slots) such as Level of Detail, Tone, and Format.
   • Reward Function: Based on Prospect Theory, the reward is a utility function that maximizes the probability of explanation acceptance. It penalizes mismatches between the prompt style and the user's latent cognitive state.
3. Policy Synthesis: Using probabilistic model checking (PRISM), COMPASS computes an optimal policy ($\sigma^*$) that maps belief states to the best prompt configuration.
4. Explanation Generation: The Prompt Generator instantiates the optimal policy into a specific prompt, which is fed to an LLM (e.g., GPT-5, Gemini, DeepSeek) to generate the final explanation.
5. Adaptation: The system continuously updates the POMDP model based on new feedback (changing acceptance rates) or context changes (e.g., a new user profile or a shift in predicted cognitive load due to time of day).

3. Key Contributions

1. Conceptual Formalization: A novel formalization of prompt generation for task planners as a POMDP, capturing both observable interactions and unobservable cognitive states.
2. COMPASS Framework: A self-adaptive approach that dynamically adjusts LLM prompts based on user feedback, cognitive predictions, and contextual changes.
3. Benchmark Dataset: A dataset of 72 tailored explanations generated for three distinct stakeholder profiles (AI experts, domain experts, non-experts) using state-of-the-art LLMs, varying in tone, detail, and format.
4. Empirical Validation: A proof-of-concept validation in two Cyber-Physical System (CPS) domains: Construction (multi-robot building automation) and Agriculture.

4. Evaluation and Results

The authors evaluated COMPASS through two case studies and user studies involving 32 participants.

• RQ1: Problem Generation Feasibility:

  • LLMs successfully generated syntactically valid and semantically interpretable JSON inputs for AI planners in both domains.
  • Limitation: While LLMs could format the input correctly, the AI planner sometimes failed to find solutions if the problem definition was incomplete (e.g., missing agent capabilities), highlighting the need for human-in-the-loop validation.

• RQ2: Adaptation Effectiveness:

  • The system successfully demonstrated runtime adaptation. For example, when a user's predicted attention dropped (late in the day), the system automatically switched the explanation from "high detail/paragraph" to "low detail/bullet points."
  • When feedback indicated a specific format was rejected by a user group, the policy updated to select a different format.

• RQ3: Alignment:

  • Perception vs. Reality: Participants generally correctly identified the tone and format of explanations. However, distinguishing between "summary" and "detailed" levels was difficult without clear reference points, suggesting a need for better prompt templates.
  • Satisfaction: Adaptive explanations were perceived as more appropriate by non-experts and AI experts compared to static ones. Domain experts showed mixed results, with some preferring the initial static format.

• RQ4: Personalisation:

  • Independent evaluators could identify the intended target profile for explanations at a rate significantly higher than random chance (approx. 50–60% accuracy vs. 33% random).
  • AI Expert explanations were the easiest to identify, while Non-expert explanations were the hardest to distinguish, likely because the underlying technical content (Pareto fronts, probabilities) remained complex even when simplified.

5. Significance and Future Work

Significance:

• Bridging the Gap: COMPASS moves beyond static XAI (Explainable AI) by introducing dynamic, cognitive-aware adaptation. It acknowledges that a "good" explanation is context-dependent.
• Reducing Cognitive Load: By modeling latent cognitive states, the system proactively reduces interaction effort for users who are fatigued or less familiar with the domain.
• Probabilistic Rigor: Using POMDPs allows the system to handle uncertainty in user states and feedback, providing a mathematically grounded approach to prompt engineering.

Future Work:

• Refining prompt instructions using zero-shot and few-shot strategies to improve clarity.
• Integrating additional planners and expanding the range of adaptive properties.
• Conducting larger, longitudinal user studies to validate the statistical significance of the adaptation benefits, particularly for domain experts.
• Improving the extraction of problem definitions to ensure complete specifications for the AI planner.

In conclusion, COMPASS demonstrates the feasibility of integrating human cognition and user feedback into automated prompt synthesis, offering a pathway toward more trustworthy and user-centric AI planning systems.