⚡ electrical engineering

ReasonSTL: Bridging Natural Language and Signal Temporal Logic via Tool-Augmented Process-Rewarded Learning

The paper introduces \textsc{ReasonSTL}, a tool-augmented framework that adapts local open-source language models via process-rewarded learning to accurately and privately translate natural language requirements into Signal Temporal Logic (STL) formulas, offering a cost-effective alternative to manual specification and commercial APIs.

Original authors: Bowen Ye, Zhijian Li, Junyue Huang, Junkai Ma, Xiang Yin

Published 2026-05-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Bowen Ye, Zhijian Li, Junyue Huang, Junkai Ma, Xiang Yin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an engineer designing a self-driving car or a robotic arm. You have a brilliant idea for how it should behave, but you can only describe it in plain English, like: "If the car gets too close to the wall, it must stop within two seconds."

The problem is that the computer's "brain" doesn't speak English. It speaks a strict, mathematical language called Signal Temporal Logic (STL). This language is like a super-precise recipe that tells the machine exactly what to do, down to the millisecond and the millimeter.

The Problem:
Until now, turning your English sentence into this strict math recipe was a nightmare.

Manual Translation: You'd need a human expert to translate it. It's slow, expensive, and hard to scale.
Asking AI (The "Black Box"): You could ask a powerful, commercial AI (like a giant cloud-based chatbot) to do it. But this is risky. You might accidentally send your company's secret safety rules to a third-party server. Plus, it costs a lot of money every time you ask a question.
Old AI Attempts: Previous local AI models were like students who memorized the alphabet but couldn't do math. They would guess the recipe, but if they got a number wrong (like saying "2 seconds" instead of "2.5 seconds"), the whole thing would fail.

The Solution: REASONSTL
The authors created a new system called REASONSTL. Think of it as hiring a local, privacy-focused apprentice who has a very specific workflow. Instead of just guessing the answer, this apprentice follows a strict three-step process:

The Translator (Reasoning): The apprentice reads your English sentence and breaks it down. "Okay, 'too close' means distance. 'Two seconds' needs to be converted to milliseconds."
The Calculator (Tool Use): Instead of guessing the math, the apprentice pulls out a calculator (a "tool"). It has specific tools for converting units (feet to meters), calculating time, and doing math. It must use these tools to get the numbers right.
The Architect (Construction): Once the numbers are verified, the apprentice builds the final math recipe (the STL formula) using a strict blueprint (a JSON tree structure) so there are no missing parentheses or confusing symbols.

How They Taught the Apprentice
They didn't just tell the apprentice, "Good job" or "Bad job" at the very end. They used a special training method called Process-Rewarded Learning.

Imagine a teacher watching the apprentice work. If the apprentice uses the calculator correctly but then builds the house upside down, the teacher says, "Good math, bad building."
If the apprentice tries to guess the math without using the calculator, the teacher stops them immediately.
This ensures the apprentice learns to think step-by-step and use the right tools, rather than just memorizing answers.

The New Test: STL-BENCH
To see if this apprentice was actually good, the team built a new, tougher test called STL-BENCH.

It's like a final exam that includes real-world scenarios (like aerospace or robotics), not just made-up sentences.
It's bilingual (English and Chinese).
It specifically tests if the apprentice can handle the "boring but critical" stuff: converting units, doing math, and understanding complex time limits.

The Results
The team tested their 4-billion-parameter model (a "small" but smart local model) against giant commercial AI models and other methods.

Privacy & Cost: Because it runs locally on their own computers, it's free to run repeatedly and keeps all data private.
Performance: Surprisingly, this local apprentice outperformed the giant commercial "black box" AI models in accuracy. It was better at getting the math right and building the correct structure.
Human Check: When humans looked at the results, the local model was just as good as the expensive commercial ones.

In a Nutshell
REASONSTL is a new way to teach local AI models to translate human safety rules into strict computer code. By forcing the AI to "show its work" using calculators and tools, and by training it to be careful at every step, they created a system that is private, cheap, and surprisingly accurate, making it a safe alternative to sending sensitive data to big cloud companies.

Technical Summary: REASONSTL

Problem Statement

Signal Temporal Logic (STL) is a formal language essential for specifying spatio-temporal requirements over real-valued signals in cyber-physical systems (CPS) and autonomous systems. While STL enables rigorous verification and synthesis, practitioners typically express requirements in natural language (NL). Translating these NL descriptions into executable STL formulas is a critical bottleneck.

Existing approaches face significant limitations:

Manual Specification: Requires expert knowledge, is labor-intensive, and does not scale.
Commercial LLM APIs: While capable, they incur high token costs and raise privacy concerns by transmitting sensitive system requirements to third parties.
Prior Automated Methods:
- DeepSTL relies on neural machine translation trained from scratch, limiting generalization.
- KGST uses a hybrid pipeline involving proprietary APIs, incurring privacy and cost issues.
- RESTL operates locally but generates formulas directly without intermediate reasoning, compromising interpretability and correctness.
Technical Challenges: NL-to-STL translation requires not just syntactic validity but also semantic grounding, computation-aware reasoning (e.g., unit conversion, arithmetic), and precise handling of temporal intervals and domain-specific signals. Small errors in thresholds or operators can lead to incorrect semantics.

Methodology: REASONSTL

The authors propose REASONSTL, a local, tool-augmented framework designed for computation-aware NL-to-STL generation. The core philosophy is to decompose the translation process into explicit reasoning, deterministic tool calls, and structured formula construction, rather than relying on single-step sequence prediction.

1. Tool-Augmented Generation

Instead of generating raw text, the model generates a structured rollout consisting of reasoning segments, tool calls, and tool outputs, culminating in a structured STL JSON tree.

Tool Set: The framework utilizes a compact, typed set of deterministic tools:
- parse_duration: Normalizes time expressions (e.g., "30 minutes" $\to$ 1800).
- convert_unit: Converts physical units (e.g., ft $\to$ m).
- eval_math_expr: Evaluates arithmetic expressions for thresholds.
- calc_time_diff: Computes time differences from timestamps.
Decoupling: This design separates linguistic interpretation and structural reasoning from exact numerical computation, making the process inspectable and verifiable.

2. Outcome-Bounded Process-Rewarded Optimization

REASONSTL introduces a novel training strategy that supervises both intermediate tool-use trajectories and the final formula construction.

Process Rewards: Intermediate stages (tool calls, reasoning) are assigned rewards based on local correctness (validity of arguments, executability, consistency).
Outcome Bounding: Crucially, intermediate rewards are capped by the correctness of the final STL formula. If the final formula is incorrect, the maximum attainable reward for intermediate steps is reduced. This prevents the model from reinforcing "plausible but semantically invalid" reasoning traces that lead to wrong final results.
Prefix Masking: If an intermediate step fails, subsequent dependent stages are masked from backward updates, ensuring the model learns to stop or correct errors early.
Optimization: The framework uses a group-relative policy objective (similar to PPO but without likelihood-ratio clipping) to optimize the model based on these bounded rewards.

3. STL-BENCH: A New Benchmark

To evaluate computation-aware NL-to-STL generation, the authors introduce STL-BENCH, a bilingual (English/Chinese) benchmark grounded in real-world CPS signals.

Features: It covers 6 engineering domains, 33 scenarios, and 41 domain-grounded signal variables.
Complexity: 73% of samples require intermediate computation (tool use). It includes explicit annotations for tool-use trajectories, temporal normalization, and unit conversion.
Structure: STL formulas are represented as JSON trees to enable schema validation and recursive structural matching, moving beyond simple string matching.
Validation: The dataset is constructed via template-anchored generation, rule-based validation, and stratified human auditing to ensure high-quality labels.

Key Contributions

Local Tool-Augmented Framework: REASONSTL is a local, open-source framework that converts NL to STL via multi-step reasoning and deterministic tool invocation, explicitly handling temporal normalization, unit conversion, and arithmetic evaluation.
Outcome-Bounded Process Supervision: A training strategy that jointly supervises intermediate tool-use trajectories and final formula construction, reducing the risk of reinforcing invalid reasoning paths by bounding intermediate rewards with final correctness.
Computation-Aware Benchmark (STL-BENCH): A bilingual benchmark featuring domain-grounded signals, physical units, arithmetic constraints, and explicit tool-use annotations, enabling rigorous evaluation of computation-aware generation.

Experimental Results

Experiments were conducted on both the established DeepSTL benchmark and the new STL-BENCH.

Performance on DeepSTL: A 4B parameter model (Qwen3-4B) trained with REASONSTL-DIRECT (a direct generation variant) achieved 81.43% Formula Accuracy and 82.57% Format Accuracy. This significantly outperformed the strongest black-box API baseline (Claude-Opus-4.7) by over 15 percentage points and improved upon the previous state-of-the-art (RESTL) by ~21 points.
Performance on STL-BENCH:
- The full REASONSTL model (with reasoning and tools) achieved 51.0% Formula Accuracy on English and 47.0% on Chinese, outperforming both the base model and carefully prompted API baselines (e.g., GPT-5.4, Kimi-K2.6).
- Human Verification: On a subset of 100 samples per language, REASONSTL matched or exceeded the performance of proprietary models, achieving 53% accuracy in English and 51% in Chinese.
- Generalization: On a manually written, template-free test set, REASONSTL achieved the highest Formula Accuracy (54%), slightly outperforming strong API baselines while maintaining higher inference throughput than some commercial models.
Ablation Studies: Results confirmed that while Supervised Fine-Tuning (SFT) helps with format initialization, Reinforcement Learning (RL) with process rewards and tool-augmented reasoning is the primary driver of performance gains.

Significance and Claims

The paper claims that REASONSTL provides a transparent, low-cost, and privacy-preserving alternative for formal specification drafting. By keeping the entire pipeline local and open-source, it addresses the privacy and cost barriers associated with commercial LLM APIs.

The authors emphasize that the framework's ability to decompose the task into deterministic computations and structured reasoning makes the generation process inspectable and amenable to validation, a critical feature for safety-critical CPS applications. While the system achieves state-of-the-art performance, the authors modestly note that it is currently best suited as a human-in-the-loop drafting assistant rather than a fully autonomous replacement for expert specification design, particularly in safety-critical contexts where residual semantic errors must be reviewed by domain experts.