SPL: Orchestrating Workflows with Declarative Deterministic-Probabilistic Composition
This paper introduces SPL (Structured Prompt Language), a declarative framework that unifies deterministic and probabilistic computation within a single specification to enable model-agnostic workflow orchestration, demonstrating through extensive experiments that its solver-based approach achieves significantly higher machine-verified correctness compared to unverified LLM-only outputs.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you're trying to build a super-smart robot assistant that can help you with your homework. Right now, building these assistants is like trying to build a car where the engine, the steering wheel, and the GPS are all made by different companies, speak different languages, and need to be glued together with messy, custom-made tape. You need to be a coding wizard just to make them talk to each other.
The paper introduces SPL (Structured Prompt Language), which is like a universal remote control that finally lets the "creative" part of the robot and the "math" part of the robot work together in a single, clean instruction manual.
The Two Brains: The Dreamer and The Calculator
The paper argues that current AI tools are stuck in one mode. They are either Dreamers (LLMs) who are great at writing stories, guessing answers, and chatting, but they sometimes make up facts or get math wrong. Or they are Calculators (like SymPy or SageMath) who are perfect at math and logic but can't understand a joke or write a story.
The authors say, "Why not have both?" They propose a system where the Dreamer (System 1) breaks a problem down and explains it, while the Calculator (System 2) does the actual heavy lifting and checks the work.
The Big Twist: The paper explicitly argues against the idea that the AI needs to be "fast" or "slow" to be one or the other. It's not about speed; it's about how they think. A calculator can be slow if it's doing a super hard proof, and a dreamer can be fast if it's just guessing. The key is knowing which brain to use for which job.
The "Design Once, Deploy Anywhere" Magic
Here is the coolest part: With SPL, you write your instructions once in a special .spl file. You don't have to rewrite the code if you want to run it on your laptop, in the cloud, or on a giant supercomputer grid.
Think of it like a recipe. You write the recipe once. Whether you cook it in a tiny camping stove (your laptop), a fancy kitchen (the cloud), or a massive industrial factory (a distributed grid), the recipe stays the same. You just tell the system where to cook it when you start. The paper calls this DODA (Design Once, Deploy Anywhere).
The "Verifier Ladder"
How do we know the math is right? The paper introduces a "Verifier Ladder" with three rungs:
- Rung 1 (SymPy): Good for basic algebra and calculus. It's fast and easy.
- Rung 2 (SageMath): For harder stuff like number theory and geometry.
- Rung 3 (Lean 4): The ultimate boss level. This is for formal proofs that are checked by a computer to be 100% mathematically true, like a legal contract for math.
The paper shows that you can write a workflow that tries Rung 1 first. If it fails, the system automatically climbs the ladder to Rung 2, and if that fails, to Rung 3. You don't have to write the "if this fails, try that" code; the language handles it for you.
The Experiment: What Actually Happened?
The authors didn't just guess; they ran a massive experiment. They tested 10 different AI models on 20 different math problems (ranging from easy to expert level) and ran each test 3 times. That's 1,200 runs in total.
They compared two ways of solving the problems:
- The "LLM-Only" Arm: The AI just guesses and writes the answer.
- The "Solver" Arm: The AI breaks the problem down, sends the math to the Calculator, gets the verified answer, and then writes the explanation.
The Results:
- The Good News: The Solver Arm was incredibly accurate. For the best models, like
gemma4:e2b, they got 93% of the answers right when verified by the calculator. Evensonnet-4-6got 85% right. - The Catch: The "LLM-Only" arm could almost always produce an answer (near 100% of the time), but it wasn't verified. The Solver Arm proved that just because an AI says something, doesn't mean it's true.
- The Bottleneck: The main reason the Solver Arm failed wasn't that the AI couldn't do the math (the Calculator did that!). It failed because the AI couldn't format its answer correctly. The AI had to write the math in a very specific code-like format (
expr|op) for the Calculator to understand. If the AI messed up the formatting, the Calculator rejected it. - The Surprise: A small, open-source model called
gemma4:e2b(which is much smaller than the giant, expensive models) actually performed better than some of the huge models at following the rules. This suggests that for this specific job, being a good "format translator" is more important than being a giant, super-smart brain.
What the Paper Says It's NOT
The paper is very clear about what it doesn't do:
- It does not claim that AI models are now perfect at math on their own. In fact, the experiment showed that without the Calculator, the models are just guessing.
- It does not say that "Thinking" models (models that spend a long time "thinking" before answering) are better. In fact, the paper excluded some "thinking" models because they spent too much time thinking and ran out of space before they could write the specific code format the Calculator needed.
- It does not claim that this solves every problem. The experiment was specifically on symbolic mathematics. The authors suggest it could work for other things like checking code or validating data, but they haven't proven that yet.
The Bottom Line
The paper proves that by separating the "creative" part of AI from the "math" part, and letting a computer check the math, we can get much more reliable results. The best part? You don't need to be a coding genius to do it. You just write the plan once, and the system handles the rest, whether you're running it on your laptop or a supercomputer.
The authors measured this with 1,200 runs and found that while the "Solver" arm is slightly slower (taking a few extra seconds to check the work), it turns a "maybe right" answer into a "machine-verified" answer. And for the best models, that verification cost almost nothing in terms of speed, proving that this two-mode approach is a practical way to build smarter, safer AI assistants.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.