Technical Summary: TwinBI – An Agentic Digital Twin for Efficient Augmented Interactions with Business Intelligence Dashboards

1. Problem Statement

Business Intelligence (BI) systems are critical for data-driven decision-making, yet a significant disconnect exists between fluent natural language generation (via Large Language Models) and analytically sound decision support. While LLM-based agents can convert natural language requests into SQL or tool executions, they often fail to maintain analytical consistency across multi-step interactions.

Specifically, when users switch between direct dashboard manipulation (e.g., filtering, drilling down) and natural language queries, the system struggles to preserve a consistent state regarding:

• Metric definitions and aggregation grains.
• Filter scopes and hierarchy levels.
• The current analytical context (e.g., which chart is active, what filters are applied).

LLM agents often drift outside the precise semantic constraints of the BI system, producing answers that are linguistically coherent but analytically inconsistent with the actual system state. Existing solutions often treat natural language and dashboard interaction as separate modes rather than a unified, synchronized workflow.

2. Methodology: The TwinBI Framework

TwinBI addresses this challenge by introducing an agentic digital-twin framework that couples an LLM-based agent system with an executable BI dashboard state. The core philosophy is to shift the role of LLM agents from "replacing BI" to "working in concert with BI" through an explicit coordination layer.

System Architecture

The system is built on a five-layer architecture:

1. Presentation Layer: Combines a chat interface (Streamlit) with embedded BI dashboards (Apache Superset). It tracks active analytical contexts (selected charts, tabs, recent interactions) and sends signals to the backend.
2. Backend Orchestration & Intelligence Layer: A FastAPI-based multi-agent system that manages dialogue history, dashboard interaction logs, and tool outputs. It routes sub-tasks to specialized agents (e.g., Schema Explorer, Chart Manager) and consolidates results anchored in the current state.
3. Semantic Layer: Built on Cube, this layer captures business meaning using declarative models of measures, dimensions, and hierarchies. It enforces compatible grains and valid joins, providing a shared semantic model for both conversational outputs and dashboard queries.
4. BI Tool Layer: Uses Apache Superset for rendering visualizations and capturing detailed interaction events.
5. Data Layer: Stores analytical datasets in DuckDB, accessed solely through the semantic layer to ensure uniform metric definitions.

Key Functional Mechanisms

• Unified Interaction Log: The system records all conversational exchanges, dashboard operations (e.g., cross-filtering, series toggling), and tool metadata. This log serves as the authoritative record for state reconstruction and provenance.
• State Reconstruction: Before interpreting a new conversational request, TwinBI encodes dashboard actions as structured events to reconstruct the current analytical state. This ensures follow-up questions are resolved against the same context as previous dashboard manipulations.
• Artifacts for Transparency: The system exposes three inspection artifacts to support user verification:
  1. Hierarchy Schema Graph: A navigable view of measures, dimensions, and hierarchies.
  2. SQL Exposure: The SQL associated with each chart query.
  3. Unified Interaction Log: Traces the relationship between conversational answers and earlier dashboard operations.
• /insights Command: A specialized command that generates state-aware summaries. It assembles a compact execution context (recent conversations, active chart, filters) and returns a summary of quantitative observations and suggested next steps, strictly grounded in the current visible evidence.

3. Evaluation and Results

The authors evaluated TwinBI through two complementary studies: a controlled A/B benchmark and a usability study.

A/B Benchmark (Agent Performance)

• Setup: Compared TwinBI against a "Dashboard-only" baseline using the same backbone agent (gpt-5-mini) and a fixed budget of 30 interaction steps across 30 analytical queries.
• Metrics: Exact-match accuracy, partial-credit accuracy, timeout rate, and interaction efficiency (steps to completion).
• Results:
  • Exact-match accuracy: Improved from 43.3% (Dashboard) to 63.3% (TwinBI).
  • Partial-credit accuracy: Improved from 48.3% to 70.8%.
  • Timeout rate: Substantially reduced from 40.0% to 10.0%.
  • Efficiency: Average steps to completion dropped from 16.47 to 6.90.
  • Behavior: TwinBI significantly reduced invalid actions and timeouts, indicating greater interaction stability. While loop-step rates remained non-trivial for some failures, the system successfully avoided the "brittle UI-only probing" that caused the baseline to fail.

Usability Study

• Setup: A within-subjects study with five participants completing three scenarios of increasing complexity (Store Performance, Product Pricing, Category Growth).
• Findings:
  • Task Accuracy: Remained high (100% in S1 and S3, 73.33% in S2), though harder scenarios required more interaction cost.
  • Workflow Patterns: Participants consistently used the dashboard to establish context first, then leveraged chat for comparison, threshold checking, or explanation. They rarely abandoned the dashboard entirely.
  • Workload: Despite increased interaction cost in complex tasks, perceived workload remained in the low-to-moderate range (NASA-TLX).
  • Feature Preference: Participants ranked state-aware combinations (e.g., "Click + Chat") higher than chat-only or direct SQL inspection. The /insights feature was useful when participants had already narrowed the analysis to a specific view.

4. Key Contributions

1. Agentic Digital Twin Framework: A novel architecture that synchronizes an LLM agent twin (modeling intent) with a BI twin (executable state), ensuring alignment between user intent and system execution.
2. Unified State Management: A mechanism to reconstruct analytical states from a unified interaction log, allowing natural language queries to be grounded in the exact filters, hierarchies, and metrics currently active in the dashboard.
3. Enhanced Transparency: The exposure of schema views, SQL, and interaction logs to support debugging and provenance tracking, moving beyond "black box" agent responses.
4. Empirical Validation: Evidence that integrating dashboard state into agent orchestration improves both analytical reliability (accuracy) and interaction stability (reduced timeouts) compared to dashboard-only or stateless agent approaches.

5. Significance and Claims

The paper claims that TwinBI demonstrates that robust "agentic BI" requires an explicit coordination layer that aligns user intent, semantic definitions, and query execution, rather than relying on natural language alone. By turning visible dashboard state into richer actionable context, TwinBI improves:

• Agent-level analytical reliability: Reducing the drift of LLM agents from semantic constraints.
• User-facing analytical support: Enabling a workflow where users can fluidly switch between direct manipulation and conversational assistance without losing context.

The authors position TwinBI as a step toward systems where agents work in concert with BI tools, enhancing the robustness of decision support. They acknowledge limitations, noting that the study involved a small sample size and a specific retail dataset, and suggest future work in scaling to larger datasets, diverse users, and transferring analytical states across different dashboards.