💻 computer science

Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines

This paper introduces a temporal semantic cache and MCP workflow optimizations to address latency and validity challenges in industrial asset operations, achieving significant speedups while highlighting the limitations of traditional semantic caching for parameter-rich, time-sensitive agentic queries.

Original authors: Alimurtaza Mustafa Merchant, Krish Veera, Sajal Kumar Goyla, Shambhawi Bhure, Dhaval Patel, Kaoutar El Maghraoui

Published 2026-05-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alimurtaza Mustafa Merchant, Krish Veera, Sajal Kumar Goyla, Shambhawi Bhure, Dhaval Patel, Kaoutar El Maghraoui

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the manager of a massive, high-tech factory. Every day, workers ask you complex questions like, "How is Machine 6 doing yesterday?" or "What went wrong with the cooling system last week?"

To answer these, you don't just guess. You have to:

Find the right tools: Locate the specific manuals and sensors for that machine.
Make a plan: Decide which steps to take first.
Do the work: Go check the sensors, read the logs, and talk to other departments.
Write the report: Summarize everything into a clear answer.

Doing this from scratch for every single question takes a long time. If a worker asks the same question in slightly different words (e.g., "How's Chiller 6?" vs. "What's the status of the 6th chiller?"), you end up doing all that work again, even though the answer is almost the same.

This paper is about building a super-smart assistant to speed up this factory manager. The authors built two main "speed boosters" to make these answers come back faster.

Booster 1: The "Time-Aware" Memory Bank

Usually, computer assistants use a "semantic cache." Think of this like a librarian who remembers questions based on how they sound. If you ask, "How is the weather?" and then ask, "What's the weather like?", the librarian says, "Oh, I just answered that!" and gives you the old answer.

The Problem: In a factory, this is dangerous.

If you ask about "Chiller 6" today, the answer is different than if you ask about "Chiller 6" yesterday.
If you ask about "Chiller 6" and the next person asks about "Chiller 9," the librarian might get confused because the sentences sound similar, but the machines are totally different.

The Solution: The authors built a Time-Aware Memory Bank. Before the librarian even looks up the answer, a smart filter checks the question:

Is it about "Right Now"? (e.g., "Is the machine broken?") → Don't use the memory bank! Go check the machine live.
Is it about "Yesterday"? → Translate "yesterday" into a specific date (e.g., Oct 12) before looking it up.
Is it about a specific machine? → Make sure the memory bank doesn't mix up Chiller 6 with Chiller 9.

The Result: When the system finds a perfect match, it skips the whole "find tools, make plan, do work" process and just hands you the answer. This made the system 31 times faster for those specific matches.

Booster 2: The "Assembly Line" Workflow

Even when the system can't use the memory bank (because the question is new), it still wastes time. In the old way, the manager did everything one step at a time: Find Tool A, then Find Tool B, then Check Sensor A, then Check Sensor B.

The Solution: The authors turned this into an Assembly Line.

Tool Discovery: Instead of waking up the tool-finding robots every single time, they kept them awake and ready. This saved a huge amount of time just finding the tools.
Parallel Work: Instead of checking Sensor A and then Sensor B one by one, they sent two workers out to check them at the same time.

The Result: Even when the system had to do the work from scratch, it finished 1.67 times faster just by organizing the workflow better.

The Big Picture: How Fast Did It Get?

When they combined both speed boosters:

The "Miss" Path: Even when the system didn't have the answer in memory, it was still 40% faster than the old way because of the Assembly Line (Booster 2).
The "Hit" Path: When the system found a cached answer, it was 31 times faster.
Overall: The average time to get an answer dropped from 34 seconds to just under 10 seconds.

The Catch (The "Failure Mode")

The paper also found a very important limitation. Even with their fancy "Time-Aware" filter, the system still got confused sometimes.

Imagine a worker asks: "How is Chiller 6 doing?"
The system finds an old answer for: "How is Chiller 9 doing?"

Because the sentences sound 95% identical, the system's "similarity check" gets tricked. It thinks, "These are the same!" and gives the wrong answer. The authors found that no matter how they tuned the system, about 36% of the time, the system couldn't perfectly tell the difference between "Chiller 6" and "Chiller 9" just by looking at the words.

The Lesson: You can't rely only on how words sound to decide if an answer is safe to reuse in a factory. You have to be very careful about the specific numbers and names (the "parameters") inside the question.

Summary

The authors built a system that:

Remembers answers but checks the date and machine name first so it doesn't give you yesterday's news for today's problem.
Organizes the work so multiple tasks happen at once instead of one by one.
Proves that while this makes things incredibly fast, you still need to be careful not to mix up similar-sounding questions about different machines.

Technical Summary: Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines

Problem Statement

Industrial asset operations workflows are highly latency-sensitive, often requiring the coordination of heterogeneous data sources including sensor telemetry, work orders, failure modes, and forecasting tools. The AssetOpsBench (AOB) benchmark formalizes this domain using a Plan-Execute paradigm where a planner LLM decomposes user queries into tool calls via the Model Context Protocol (MCP), and an executor invokes these tools.

This two-stage structure introduces substantial wall-clock latency due to repeated overhead in:

Tool Discovery: Spawning servers and fetching tool signatures for every query.
LLM Planning: Converting queries and catalogs into structured plans.
MCP Execution: Sequentially invoking tools.
Summarization: Synthesizing outputs.

While standard LLM caching techniques (KV-cache reuse, embedding-based semantic caching) exist, they fail in this industrial context. Context caching relies on identical prompt prefixes, which is rare in paraphrased operational queries. Semantic caching assumes output validity depends solely on the input prompt, ignoring that industrial outputs depend on external, time-varying states (e.g., asset status, sensor readings, time windows). Consequently, pure semantic similarity often leads to stale or incorrect answers when parameters (asset ID, time window) shift, even if the linguistic structure remains similar.

Methodology

The authors propose two complementary optimization layers for AOB pipelines: a Temporal Semantic Cache and MCP Workflow Optimizations.

1. Temporal Semantic Cache

This layer extends standard semantic caching to handle the time-sensitivity and parameter-dependency of industrial queries.

Temporal Classification: Before semantic retrieval, a lightweight regex-based classifier routes queries into four buckets:
- Volatile: Requests for live state (bypasses cache).
- Static: No temporal dependence (standard semantic match).
- Relative: Expressions like "yesterday" resolved into concrete windows at insertion time.
- Anchored: Fixed time windows matched against compatible windows.
Two-Stage Retrieval:
1. Candidate Retrieval: Uses Approximate Nearest Neighbor (ANN) search on query embeddings (Qwen3).
2. Judging: A reranker-based judge evaluates candidates for both semantic alignment and window compatibility. This prevents returning cached answers from different time periods or asset contexts.
Resolution Strategy: Relative time phrases are resolved to concrete timestamps upon insertion. The judge checks if the cached window is compatible with the new query's resolved window.

2. MCP Workflow Optimizations

These optimizations reduce orchestration overhead regardless of cache state.

Disk-Backed Tool-Discovery Caching: Instead of spawning subprocesses and calling list_tools() for every query, the tool catalog is persisted to a local JSON file. The cache key invalidates only on changes to server source code or configuration.
Dependency-Aware Parallel Execution: The generated plan is treated as a Directed Acyclic Graph (DAG). Steps are grouped into topological dependency layers. Independent steps within a layer execute concurrently using a persistent server pool (maintaining stdio sessions), while dependency barriers preserve ordering. Failures in one server do not halt sibling steps on other servers.

Key Contributions

Temporal Semantic Caching for Industrial Agents: The introduction of a pre-retrieval temporal classifier and a window-aware judge to address parameter- and time-sensitivity, moving beyond pure semantic similarity.
MCP Workflow Optimizations: A combination of discovery-phase caching, DAG-layered parallel execution, and persistent server pools to reduce per-query orchestration costs.
Refined Evaluation Setup: A paired baseline-vs-optimized harness with a paraphrase-tier test set using parent_id-based ground truth for cache hit/miss labeling, enabling systematic ablation studies on a single machine.
Critical Analysis of Caching Validity: An empirical demonstration that pure semantic similarity is an unsound proxy for answer validity in parameter-rich industrial queries, capping hit-decision F1 scores due to "parameter-collision" false positives.

Results

Experiments were conducted on the AOB benchmark (152 queries) using an Apple M-series machine.

MCP Workflow Gains (Isolated): On 18 IoT queries, MCP optimizations alone yielded a 1.67× end-to-end speedup.
- Discovery cost reduced by 296× (from ~2.1s to ~0.007s).
- Execution time reduced by 1.99× via parallelization.
- Planning and summarization times remained largely unchanged, confirming the optimizations target orchestration, not LLM inference.
Combined Pipeline: On 80 paraphrase-tier queries, the full pipeline achieved a 3.48× median speedup (latency reduced from 34.10s to 9.80s).
- Cache Hits: Achieved a 31.87× speedup (median latency near zero) for 45% of queries.
- Cache Misses: The miss path remained 3.30s faster than the unoptimized baseline, proving the MCP optimizations provide additive benefits even when the cache misses.
Cache Decision Quality: The temporal-classifier-plus-judger achieved an F1 of 0.64 on hit/miss decisions.
- Precision: 0.75
- Recall: 0.56
- Specificity: 0.72

Significance and Limitations

The paper argues that while the proposed optimizations significantly reduce latency, the primary contribution is a critical analysis of caching as an evaluation choice.

Failure Mode Analysis: The study exposes a structural limitation: pure semantic similarity cannot distinguish between queries that differ only in operational parameters (e.g., "Chiller 6" vs. "Chiller 9") but share the same linguistic frame. Even with a strict reranker threshold ( $\tau_{judge} = 0.92$ ), "parameter-collision" false positives occur, capping the F1 score near 0.64. This suggests that for parameter-rich industrial queries, semantic caching alone is not a safe proxy for answer validity without parameter-aware extensions.
Additivity: The two layers are independent and additive; the workflow optimizations ensure that the "miss path" is always faster than the baseline, preventing the cache from introducing overhead.
Modest Claims: The authors acknowledge that the current temporal classifier handles a fixed grammar of relative phrases, missing some natural date expressions. They also note that the evaluation was limited to a single machine with no persistence, and the paraphrase-tier test set was biased toward warm parents, resulting in a higher observed hit rate (45%) than likely achievable on the full, unfiltered corpus (estimated 15–30%).

The work concludes that while significant speedups are achievable, future work must focus on parameter-aware caching (extracting structured parameters for exact matching) to resolve the structural failure modes identified in industrial agent benchmarks.