Technical Summary: Reflecti-Mate

Problem Statement

High-stakes personal decision-making involves a complex interplay of cognitive, emotional, and intuitive processes. While integrating these diverse sources of judgment leads to more effective decisions, individuals often struggle to do so due to limited metacognitive awareness. Current decision-support systems predominantly focus on cognitive reasoning (System 2), often treating emotional and intuitive inputs (System 1) as heuristic biases to be minimized or ignored. Consequently, existing adaptive systems tend to produce homogenized interactions that steer all users toward cognitive elaboration, failing to respect individual thinking profiles or support the integration of non-cognitive perspectives. There is a lack of systems that model user thought patterns over the course of a reflective dialogue to adaptively support integrative reasoning without imposing a single "ideal" reasoning style.

Methodology

System Architecture: Reflecti-Mate

The authors propose Reflecti-Mate, a conversational agent designed to support integrative pre-decisional reflection. The system operates on a dual-process framework, aiming to apply System 2-level elaboration to all thought types (internal/emotional, external/cognitive, and experiential/intuitive) without privileging one over the others.

1. User Modeling and Reflection Profile:
The system constructs a computational representation of the user's reflection, denoted as $R = (T, G)$, where $T$ is a set of main thoughts and $G$ represents parent-child relations between thoughts and their elaborations.

• Thought Categories: Thoughts are classified into four categories based on the "head-heart-gut" framework: Internal (feelings, values), External (factual constraints, stakeholders), Experiential (past experiences), and Other.
• Metrics: The agent calculates two key dimensions for each category $k$:
  • Breadth ($B_k$): The number of distinct main thoughts in a category.
  • Depth ($D_i$): The elaborative richness of a thought, defined as $1 + |E_i|$ (where $E_i$ is the set of elaborations).
  • Fixation Indicator ($S_k$): A cumulative measure of attention on a category, combining breadth and depth.
  • Average Depth ($\bar{D}_k$): The average elaboration level within a category, independent of the number of thoughts.

2. Adaptation Algorithm:
The agent employs an exploration-exploitation tradeoff strategy guided by an internal state $s_t$ that updates after every turn.

• Disrupting Fixation (Exploration): If a user fixates on a specific category (high $S_k$), the agent prioritizes exploration by asking questions targeting underrepresented categories (analogous to uncertainty-based sampling).
• Disrupting Shallow Reasoning (Exploitation): If thoughts are surface-level (low $\bar{D}_k$), the agent prioritizes exploitation by generating Socratic follow-up questions to deepen specific underdeveloped thoughts.
• Decision Policy: The agent uses an $\epsilon$-greedy policy to select between exploration (probability $\epsilon$) and exploitation (probability $1-\epsilon$).
• Implementation: The system utilizes a locally hosted Phi-4 14B model (Q4_K_M quantization) for generating Socratic follow-ups and categorizing user inputs. Exploration questions are drawn from a curated, peer-reviewed set to control for phrasing variables.

User Study Design

A between-subjects study ($N=128$) was conducted to compare the experimental agent (Reflecti-Mate) against a Baseline Agent.

• Procedure: Participants engaged in two phases: (1) Unaided Reflection, where they articulated thoughts freely; and (2) Assisted Reflection, where they interacted with the assigned agent.
• Conditions:
  • Experimental: The agent used the adaptive profile and algorithm described above.
  • Baseline: The agent shared the same high-level objectives (broaden and deepen reflection) and underlying LLM but lacked the explicit user profile and adaptive logic, relying only on conversational history and fixed instructions.
• Measures:
  • Reflective Language: Analyzed using LIWC-22 to quantify cognitive, emotional, and intuitive linguistic markers.
  • Perceived Support: Assessed via Likert scales regarding holistic integration and elaboration depth.
  • Heterogeneity: Post-hoc clustering analyzed whether effects varied by users' initial reflective profiles (Reserved, Intuition-dominant, Emotion-dominant).

Key Results

1. Reflective Language and Integration (RQ1)

The experimental agent successfully fostered a more integrative reflective language profile compared to the baseline.

• Baseline: Produced a homogenized profile dominated by cognitive linguistic markers, suppressing emotional and intuitive expression relative to the baseline unaided phase.
• Experimental: Facilitated multi-modal participation. Users exhibited significantly lower cognitive dominance and higher expression of emotional and intuitive markers compared to the baseline. Crucially, cognitive markers remained robust; the agent did not suppress cognition but enabled emotional and intuitive markers to coexist at comparable levels.

2. Perceived Support (RQ2)

Participants perceived the experimental agent as significantly more effective at supporting holistic integration (72% agreement vs. 44% in baseline). However, both agents were rated highly for elaboration depth (72% vs. 79%), indicating that while the baseline could deepen thoughts, it failed to integrate diverse thought types.

3. Preservation of Reasoning Styles (RQ3)

The study investigated whether the agent preserved users' dominant reasoning styles.

• Baseline Effect: The baseline agent acted as a "homogenizer." Regardless of whether users started as Reserved, Intuition-dominant, or Emotion-dominant, the baseline interaction shifted all groups toward a similar, cognition-heavy profile.
• Experimental Effect: The adaptive agent produced heterogeneous effects. It selectively expanded underrepresented modes while maintaining the prominence of the user's initially dominant mode. For example, Intuition-dominant users remained intuition-dominant but gained scaffolding in emotion; Emotion-dominant users remained emotion-dominant but gained scaffolding in intuition. This demonstrates that the agent supported personalized trajectories without forcing convergence to a single reasoning style.

Significance and Contributions

The paper claims that Reflecti-Mate offers a novel approach to decision support by moving beyond "one-size-fits-all" cognitive nudging. Its primary contributions are:

1. Adaptive Integration: It demonstrates that conversational agents can support the integration of cognitive, emotional, and intuitive processes by dynamically balancing exploration and exploitation based on real-time user modeling.
2. Respect for Individuality: Unlike prior systems that often equate reflection support with rationalization, this approach respects diverse reasoning preferences. It allows users to maintain their dominant reasoning style while broadening their perspective, avoiding the imposition of a single "ideal" profile.
3. Computational Awareness: The study highlights that effective reflection support requires principled adaptation grounded in structured user modeling, rather than relying solely on prompt engineering or conversational history. Instructions alone were insufficient to prevent the baseline agent from homogenizing user responses.
4. Ethical Design: By avoiding directive advice and focusing on facilitative inquiry, the system aims to preserve user agency and autonomy in high-stakes decisions, addressing the risk of LLMs overly influencing or biasing user decisions.

The authors conclude that while the system successfully elicited integrative language and was perceived as supportive, future work is needed to explore personalized exploration questions and to validate these findings with longitudinal studies and behavioral measures beyond linguistic proxies.