Engineering Systems for Data Analysis Using Interactive Structured Inductive Programming

The paper introduces iProg, an interactive tool that leverages a structured communication protocol between humans and large language models to decompose scientific data analysis tasks into declarative Data Flow Diagrams and generate corresponding code, thereby achieving significantly faster development, higher code quality, and better performance than traditional Low Code/No Code alternatives.

The Gist

Imagine you are a master architect (the human engineer) who needs to build a complex, custom house (a scientific data analysis system). You have a very talented but sometimes scatterbrained assistant (the Large Language Model, or LLM) who can write blueprints and lay bricks incredibly fast.

The problem? If you just tell the assistant, "Build me a house," they might give you a castle with a swimming pool in the roof, or a shed with no doors. If you try to fix it by just chatting back and forth, you might end up with a Frankenstein's monster of a building that looks okay from the outside but collapses when you turn on the lights.

This paper introduces iProg, a new way to work with AI assistants that solves this problem. Think of iProg not as a "magic button" that builds the house for you, but as a strict, structured communication protocol that forces the human and the AI to work together like a perfect team.

Here is how it works, broken down into simple steps:

1. The Two-Step Dance: Structure First, Bricks Second

Most "No-Code" tools try to build the whole house in one giant leap. iProg says, "No way. Let's do this in two distinct phases."

• Phase 1: The Blueprint (The DFD)
  Before laying a single brick, the human and the AI sit down to draw a Data Flow Diagram (DFD). Imagine this as a flowchart or a subway map.

  • The AI says: "Here is my idea: We need a station for 'Data Loading,' a tunnel for 'Cleaning,' and a terminal for 'Analysis'."
  • The Human says: "Wait, that tunnel is too short. Let's add a stop for 'Validation' here."
  • They keep swapping ideas until they both Ratify (agree) on the map.
  • Why this matters: If you get the map wrong, the whole house is wrong. iProg ensures the map is perfect before any code is written.

• Phase 2: Laying the Bricks (Code Generation)
  Once the map is approved, the AI starts building one room at a time.

  • The Human points to the "Data Loading" room on the map and says, "Okay, build this. Here are the rules: It must start with a CSV file and end with a clean table."
  • The AI builds the room.
  • The Human inspects it. "Good, but the door is the wrong color." -> Revise.
  • The AI fixes it. "Perfect." -> Ratify.
  • They move to the next room.

2. The "Traffic Light" System (The Protocol)

The secret sauce of iProg is a strict set of rules for how the human and AI talk. They don't just chat; they use specific "tags" like traffic lights:

• 🟢 RATIFY: "This is good. Move on."
• 🔴 REFUTE: "This is wrong. Explain why or try again."
• 🟡 REVISE: "I see what you tried, but change this specific part."
• ⚫ REJECT: "Stop. This entire approach is broken."

This prevents the AI from rambling or the human from getting frustrated. It forces the AI to stop and think before it writes a single line of code.

3. The Results: Faster, Better, and Safer

The authors tested this on two real-world scientific projects: one about stars (Astrophysics) and one about proteins (Biochemistry).

• The "No-Code" AI (The Wild West): When they let the AI just try to build the whole system without the map or the traffic lights, it failed. It produced code that didn't run, or systems that gave wrong answers. It was like trying to build a house by throwing bricks at a wall and hoping they stick.
• The "Low-Code" AI (The Unstructured Helper): When they let the AI build it with some human help but no strict rules, it was okay for simple tasks but failed on complex ones.
• The iProg Approach (The Structured Team):
  • Speed: It was 10 times faster than hiring a team of humans to do it manually.
  • Quality: The code was cleaner, easier to read, and had fewer errors than the "No-Code" attempts.
  • Accuracy: The final results were actually better than the original systems built by human teams over months.

The Big Picture Takeaway

This paper argues that we shouldn't replace humans with AI, nor should we let AI run wild.

Instead, we should use AI as a powerful junior engineer who needs a senior architect (the human) to provide the structure. By forcing the AI to draw the map first and then build room-by-room with strict approval checks, we get the speed of AI with the reliability of human engineering.

In short: iProg turns a chaotic conversation with a super-smart but confused robot into a disciplined, step-by-step construction project that builds high-quality scientific tools in record time.

Technical Summary

Here is a detailed technical summary of the paper "Engineering Systems for Data Analysis Using Interactive Structured Inductive Programming."

1. Problem Statement

The development of information systems (IS) for scientific data analysis faces a critical bottleneck: balancing the need for correctness, maintainability, and interpretability with the speed of development.

• Manual Development: Reliable but slow and resource-intensive, requiring sustained collaboration between software engineers and domain specialists.
• Automated "No Code" / LLM Approaches: Fast but often unreliable. Large Language Models (LLMs) struggle with multi-step reasoning, workflow design, and integration. They frequently produce code that lacks structural integrity, contains runtime errors, or fails to meet complex domain specifications when prompted with unstructured natural language.
• The Core Challenge: Determining an appropriate system decomposition (breaking a complex problem into sub-tasks) before code generation can occur. Current LLMs fail to reliably perform this structural planning without human oversight.

2. Methodology: iProg

The authors propose iProg, a tool implementing Interactive Structured Inductive Programming (ISIP). It bridges the gap between manual engineering and full automation by using a "2-way Intelligibility" communication protocol to constrain LLM interactions. The process is divided into two coupled stages:

A. Interactive Structure Identification (iStruc)

Instead of generating code immediately, iProg first asks the LLM to decompose the natural language problem description into a Data Flow Diagram (DFD).

• Representation: The system is modeled as a Directed Acyclic Graph (DAG) where nodes are processes (sub-tasks) and edges are data flows.
• Declarative Specification: Each process node includes:
  • Description: Natural language specification.
  • Pre-condition: Required state/data before execution.
  • Post-condition: Guaranteed state/data after execution.
• Interaction Protocol: The LLM proposes a DFD. The human engineer responds using specific tags:
  • RATIFY: Accepts the structure.
  • REFUTE: Rejects specific parts (e.g., granularity or interface mismatches).
  • REJECT: Stops the attempt if the proposal is fundamentally flawed.
  • This loop continues until a ratified DFD is achieved.

B. Interactive Code Generation (iCode)

Once the DFD is ratified, iProg generates code for each process node individually.

• Component-wise Synthesis: The system traverses the DFD (breadth-first) and invokes the LLM to generate code for each sub-task based on its specific pre/post-conditions.
• Verification Loop: Similar to structure identification, the human reviews the generated code and explanation. The interaction uses the same tagging protocol (RATIFY, REFUTE, REVISE).
• Composition: Verified sub-programs are composed into an end-to-end executable system based on the DFD connectivity.

C. The "2-way Intelligibility" Protocol

The core innovation is the formalization of human-LLM interaction. Unlike open-ended dialogue, this protocol enforces:

1. Explicit Tags: Messages carry semantic tags (RATIFY, REFUTE, etc.) rather than just text.
2. Asymmetric Roles: Humans can REFUTE or REJECT but do not REVISE (they reject the LLM's output, forcing the LLM to revise). Machines never REJECT (they always attempt to provide a solution).
3. Context Management: The system maintains a context of previous interactions to guide the LLM toward convergence.

3. Key Contributions

1. Methodology for Semi-Automated IS Engineering: A novel framework that uses LLMs to identify system decompositions (DFDs) from natural language and generate human-ratified code, treating scientific data analysis as workflow-oriented development.
2. The iProg Tool: A functional implementation that supports interactive DFD identification, manual DFD editing, and component-wise code generation with verification.
3. Empirical Validation: Extensive evaluation on two real-world scientific domains (Astrophysics and Biochemistry), demonstrating that structured induction outperforms both manual development and standard "Low Code/No Code" LLM approaches.

4. Results

The study evaluated iProg against Manual Development (original published systems) and Baselines (No Code: direct LLM generation; Low Code: LLM with free-form human feedback).

• System Performance (Accuracy):
  • Astrophysics (PHY): iProg achieved significantly lower error rates (0.020–0.030) compared to Low Code baselines (0.133–0.357) and even outperformed the original manual system (0.058–0.164).
  • Biochemistry (BIO): iProg achieved an error rate of 0.024, comparable to the manual system (0.021). Low Code baselines failed completely (runtime errors or non-functional systems).
• Code Quality:
  • iProg generated structured, modular code with explicit interfaces.
  • Logic Scores: iProg scored 6.69/10 on logic checks (pylint), whereas Low Code baselines scored 1.71–1.88.
  • Low Code approaches produced monolithic, brittle code with high runtime error rates.
• Development Effort:
  • Time: iProg reduced development time from 30–60 days (manual, 2–3 engineers) to 4–10 days (1 engineer).
  • Efficiency: iProg required similar interaction counts to Low Code baselines but yielded vastly superior results due to the structural constraints.
• Structure Learning: iProg successfully identified DFDs that aligned with expert domain workflows, requiring only minor refinements (3–4 interactions for structure, 13–22 for code).

5. Significance and Conclusion

The paper demonstrates that Interactive Structured Inductive Programming is a viable path for engineering complex scientific data systems.

• Human-in-the-Loop is Essential: The study confirms that while LLMs can propose meaningful structures and code, human verification via a structured protocol is necessary to ensure correctness and convergence.
• Structure Enables Quality: By forcing the LLM to define a DFD with pre/post-conditions before coding, the system avoids the "hallucination" and integration failures common in direct code generation.
• Scalability: The approach allows for rapid iteration; if a parameter changes (e.g., in the Biochemistry model), only the specific process node needs re-generation, leaving the rest of the validated system intact.

Conclusion: iProg offers a "best of both worlds" solution: the speed of LLM-assisted generation combined with the rigor of traditional software engineering, resulting in systems that are faster to build, more reliable, and easier to maintain than current Low Code/No Code alternatives.