💻 computer science

OpenQlaw: An Agentic AI Assistant for Analysis of 2D Quantum Materials

OpenQlaw is an agentic AI assistant that orchestrates a physics-aware multimodal model and a lightweight framework to decouple visual identification from reasoning, enabling dynamic, context-aware analysis of 2D quantum materials that accelerates device fabrication through persistent memory and naturalistic interaction.

Original authors: Sankalp Pandey, Xuan-Bac Nguyen, Hoang-Quan Nguyen, Tim Faltermeier, Nicholas Borys, Hugh Churchill, Khoa Luu

Published 2026-03-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sankalp Pandey, Xuan-Bac Nguyen, Hoang-Quan Nguyen, Tim Faltermeier, Nicholas Borys, Hugh Churchill, Khoa Luu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to find a specific type of rare gemstone hidden inside a massive, messy pile of rocks. You have a super-smart assistant (an AI) who can look at a photo of the pile and tell you exactly where the gems are.

The Problem:
In the world of quantum materials (tiny, flat sheets of stuff used for future computers), scientists face this exact problem. They take photos of these materials under a microscope to find the perfect "flakes" (the gems).

However, the current AI assistants are like over-enthusiastic law students. When you ask them, "Where is the biggest gem?", they don't just point to it. Instead, they write a 10-page essay explaining why they think it's the biggest, listing every single rock's coordinates in a giant, confusing spreadsheet, and then finally giving you the answer.

This is great for understanding how they think, but terrible for a scientist standing in a lab who just needs to know: "Which one is it, how big is it in real life, and can you draw a circle around it so I can find it?" The scientist gets "cognitive overload"—too much information, not enough action.

The Solution: OpenQlaw
The researchers built OpenQlaw, which acts like a smart project manager or a concierge for these scientists.

Here is how it works, using a simple analogy:

1. The Team Structure

Think of OpenQlaw as a small office with three key roles:

The Expert (QuPAINT): This is the "Law Student" mentioned earlier. It is a super-smart AI that knows the physics of these materials. It can look at a photo and identify every single flake, but it talks too much and gives raw data that is hard to use directly.
The Manager (The Orchestrator): This is the new "Agent." It talks to the scientist via simple apps like WhatsApp or Discord. It doesn't try to do the heavy physics lifting itself. Instead, it acts as a filter.
The Tools (The Workers): These are simple, precise calculators and drawing tools that do exactly what they are told without making mistakes.

2. How the Conversation Flows

Imagine a scientist sends a photo to the Manager via WhatsApp and says: "Find the biggest single-layer flake and tell me its area."

Step 1: The Handoff. The Manager doesn't try to guess. It quietly asks the Expert to analyze the photo.
Step 2: The Filter. The Expert comes back with that long, messy list of coordinates and a 500-word explanation. The Manager reads this, ignores the fluff, and grabs the specific numbers it needs (like the width and height of the flake).
Step 3: The Calculation. The Manager takes those numbers and hands them to the Calculator. If the scientist previously said, "1 pixel equals 0.25 micrometers," the Manager remembers this! It calculates the real-world size (e.g., "This flake is 12.21 micrometers wide").
Step 4: The Visual. If the scientist asks, "Show me that flake," the Manager tells the Drawing Tool to take the original photo and draw a neat, clean circle around just that one flake. It doesn't clutter the image with circles around every other rock.
Step 5: The Answer. The Manager sends a simple, natural message back to the scientist: "The largest flake is located at these coordinates. It is 12.21 x 9.23 micrometers. Here is the image with it highlighted."

3. The "Memory" Trick

One of the coolest features is the Persistent Memory.
In the old way, if you asked the AI to measure a flake, then asked it to measure another one five minutes later, you'd have to tell it the microscope scale again.
OpenQlaw is like a good secretary. It remembers: "Oh, you told me earlier that this microscope's scale is 0.25 micrometers per pixel." It saves this info and uses it for all future calculations in that conversation, so the scientist never has to repeat themselves.

Why This Matters

Before OpenQlaw, a scientist had to:

Get a photo.
Run a complex AI model.
Read a confusing text dump.
Manually copy numbers into a calculator.
Manually draw on the image.
Repeat for every single flake.

With OpenQlaw, it's a one-step conversation. The scientist gets a clear, actionable answer instantly. It turns a slow, boring, manual process into a fast, automated workflow, allowing scientists to build better quantum computers faster.

In short: OpenQlaw takes a smart but chatty AI, puts it in a room with a strict manager and some precise tools, and gives the scientist a simple phone number to call for instant, useful results.

1. Problem Statement

The characterization of two-dimensional (2D) quantum materials (e.g., graphene, MoS₂, hBN) is critical for next-generation electronics but faces significant bottlenecks in the transition from optical identification to device fabrication.

Visual Ambiguity: Standard optical microscopy struggles to distinguish between monolayers, bilayers, and few-layer flakes due to subtle visual differences.
Workflow Inefficiency: Researchers currently rely on a slow, manual workflow involving optical identification followed by Atomic Force Microscopy (AFM) for validation. This requires physically relocating samples and remapping coordinates, creating a throughput bottleneck.
Limitations of Current AI:
- Traditional Computer Vision: Object detectors and segmentation models lack "physical priors" (e.g., thin-film interference physics), leading to errors in phase identification under varying illumination or substrate conditions.
- Multimodal Large Language Models (MLLMs): While models like QuPAINT (Physics-Aware Instruction Tuned) successfully reason about physical properties, their outputs are optimized for cognitive transparency rather than execution. They produce verbose, dense reasoning traces and raw coordinate enumerations that induce cognitive overload and lack immediate utility for researchers needing specific metrics (e.g., area calculations) or visual annotations.

2. Methodology: The OpenQlaw Framework

OpenQlaw is introduced as a multi-agent orchestration system designed to bridge the gap between specialized MLLMs and practical laboratory decision-making. It decouples visual identification from deterministic computation and conversational interaction.

Architecture Components

The system is built upon NanoBot (an agentic framework inspired by OpenClaw) and integrates three primary components:

Agentic Orchestration Loop (The Core Agent):
- Acts as the central reasoning engine, powered by Qwen3-VL-32B-Instruct.
- Manages user interactions via standard messaging channels (WhatsApp, Discord), allowing researchers to upload images directly from mobile devices.
- Persistent Memory: Maintains session context, storing spatial metadata (e.g., microscope scale factors like $1 \text{ pixel} = 0.25 \mu m$ ) and sample preparation methods across multi-turn conversations.
- Autonomous Inference: Can infer measurement units by analyzing physical scale bars in uploaded images.
Material Domain Expert (QuPAINT):
- Operates strictly as a specialized tool/node, not a direct chatbot.
- Performs heavy, physics-aware inference to identify flakes, distinguish layers, and output raw bounding box coordinates $[x, y, w, h]$ alongside a detailed reasoning trace.
- The orchestrator intercepts this output, extracting only the structured numerical data while suppressing the verbose text.
Deterministic Execution Tools:
- Mathematical Tool: Converts pixel coordinates to physical metrics using the formula $Area_{physical} = (w \times S) \times (h \times S)$ , where $S$ is the scale factor.
- Image Annotation Tool: Uses the Python Imaging Library (Pillow) to dynamically render images, drawing isolated bounding boxes or highlighting specific flakes based on user constraints (e.g., "Show me the largest monolayer").

Workflow

Input: Researcher uploads an image and a natural language query (e.g., "Find the largest monolayer and calculate its area").
Delegation: The Orchestrator calls QuPAINT via API to analyze the image.
Parsing: The Orchestrator extracts raw coordinates from QuPAINT's output.
Execution:
- If scale data exists in memory, the Math Tool calculates the physical area.
- If the user requests visualization, the Annotation Tool generates a modified image.
Output: The system returns a concise, natural language response with actionable metrics and/or a visual overlay, bypassing the need for the user to parse dense text.

3. Key Contributions

Agentic Orchestration: Introduces a framework that separates the "expert" (QuPAINT) from the "assistant," allowing the core agent to manage context, memory, and tool execution while the expert focuses solely on physics-informed visual reasoning.
Decoupling Reasoning from Execution: Solves the "cognitive overload" problem of MLLMs by filtering verbose reasoning traces and converting raw data into deterministic, actionable metrics (physical area, coordinates).
Persistent Context & Memory: Implements a memory system that retains physical scale ratios and sample preparation methods, enabling accurate calculations across multiple queries without redundant data entry.
Accessibility: Deploys the system via standard messaging APIs (WhatsApp/Discord), enabling real-time, mobile-first interaction for lab floor researchers without dedicated software overhead.

4. Results and Discussion

The paper presents a qualitative case study comparing QuPAINT (Baseline) vs. OpenQlaw:

Baseline (QuPAINT): When queried to count flakes, the model correctly identifies them but outputs a dense, raw enumeration of coordinates. It often miscounts the final number due to LLM limitations with long numerical strings. The output is text-heavy and lacks a mechanism to determine when visualization is actually needed, wasting compute resources.
OpenQlaw:
- Conciseness: Provides direct answers (e.g., "The largest monolayer is $12.21 \times 9.23 \mu m^2$ ") without exposing the intermediate reasoning trace.
- Accuracy: Successfully calculates physical areas by applying cached scale factors, avoiding the "arbitrary units" error seen in the baseline.
- Dynamic Rendering: Only generates visual annotations when explicitly requested, conserving resources.
- Memory Utilization: Demonstrated the ability to recall scale factors from previous turns to convert pixel counts into micrometers ( $\mu m^2$ ) accurately.

5. Significance and Future Work

Significance:
OpenQlaw transforms isolated AI inferences into a context-aware assistant capable of accelerating high-throughput device fabrication. By mitigating cognitive overload and providing immediate, actionable physical metrics, it bridges the gap between advanced AI research and practical laboratory workflows.

Limitations:

Token Constraints: Reliance on QuPAINT means extremely dense images with hundreds of flakes may exceed context windows, leading to truncated coordinate arrays.
Latency: Maintaining persistent memory and running a 32B-parameter local model introduces computational overhead and inference latency.
Dependency: The system is brittle to formatting errors; if the Domain Expert hallucinates or deviates from the expected $[x, y, w, h]$ format, downstream Python scripts may fail.

Future Work:
The authors plan to expand the toolset to include direct hardware interfacing, allowing the agent to autonomously adjust microscope settings based on visual feedback or control robotic systems to fully automate the exfoliation, flake hunting, and stacking pipeline.