💻 computer science

ScratchLens: Lens-Parametric Behavioral Equivalence for Scratch Programs

ScratchLens introduces a lens-parametric framework for determining behavioral equivalence in Scratch programs by compiling them into a causal intermediate representation, applying canonicalization and partial-order reduction to handle concurrency, and utilizing SMT-based refinement to provide sound verdicts with evidence, thereby overcoming the limitations of syntactic differencing and single-run dynamic analysis in automated grading and repair.

Original authors: Yuan Si, Jialu Zhang

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuan Si, Jialu Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a teacher grading a class of students who are learning to code using Scratch, a colorful, block-based programming language. Two students, Alice and Bob, submit projects that look completely different on the screen. Alice renamed all her variables, split her code into tiny pieces, and rearranged her instructions. Bob kept his code simple but changed one tiny block.

The big question for the teacher is: Are these two programs actually doing the same thing, or is one of them broken?

This is where the paper introduces ScratchLens. Think of ScratchLens as a super-smart, magical magnifying glass that doesn't just look at the words (the code blocks) but understands the story (the behavior) behind them.

Here is how the paper explains this tool, broken down into simple concepts:

1. The Problem: "Same Story, Different Chapters"

In Scratch, you can tell the same story in a million different ways.

The "Renaming" Trick: If Alice calls a variable score and Bob calls it points, a simple computer program might think they are totally different. But a human knows they are the same.
The "One-Block" Trap: Sometimes, changing just one tiny block (like removing a "wait" command) can break the timing of the program. The program might still look like it works, but if you watch closely, the characters might move out of sync.

Existing tools are bad at this. Some are too strict (they say "different!" just because the code looks different). Others are too loose (they run the program once, see it works, and say "same!" even if it's actually broken).

2. The Solution: The "Lens" Concept

The authors realized that "equivalence" depends on what you are looking at. They call this a Lens.

Imagine looking at a movie through different colored glasses:

The "Final State" Lens: You only care if the game ends with the same score. (Did the player win?)
The "Frame" Lens: You care about the animation. (Did the character glide smoothly or jump instantly?)
The "Event" Lens: You care about the timing. (Did the character wait for the music to start before dancing?)

ScratchLens doesn't just give a "Yes/No" answer. It says: "Under the Final State lens, these are the same. But under the Frame lens, they are different because one jumps and the other glides." This prevents the tool from giving a confusing answer when two people are looking at the problem differently.

3. How It Works: The "Recipe" vs. The "Meal"

Instead of just comparing the list of ingredients (the code blocks), ScratchLens cooks the meal and analyzes the recipe in a special way:

The "Causal Map" (CSIR): It translates the colorful blocks into a strict, mathematical map of cause-and-effect. It tracks who reads what, who writes what, and who waits for whom.
The "Magic Sort" (Canonicalization): It takes the code and reorganizes it into a standard format. If Alice wrote A + B and Bob wrote B + A, the tool knows these are the same math problem. It strips away the "noise" (renaming, reordering) to see the core logic.
The "Race Detector": In Scratch, many things happen at the same time. The tool checks if two actions are fighting over the same resource (like two people trying to write on the same whiteboard). If they are, it flags it as a potential "race condition" (a timing bug).

4. The "Detective" Mode

If the tool isn't 100% sure, it doesn't just guess. It acts like a detective with a Counter-Example Generator:

It says, "I think these are different, but I need proof."
It then runs a specific, targeted test (like a "stress test") to see if it can force the programs to behave differently.
If the test fails, it catches the bug and shows the teacher exactly why (e.g., "Missing a 'wait' command caused the character to move too early").
If it still can't prove a difference, it honestly says, "Unknown." This is crucial. It's better to say "I don't know" than to lie and say "They are the same" when they might be broken.

5. The Results: The "Perfect Score"

The authors tested ScratchLens on a massive collection of real student projects and "mutated" versions (where they intentionally broke the code in specific ways).

The Test: They had 444 pairs of programs to compare.
The Result: ScratchLens got 100% accuracy.
The "False Positive" Safety: Most importantly, it never claimed a broken program was correct. In the world of grading, saying a broken program is "good" is the worst mistake. ScratchLens avoided this completely.
Comparison: Other methods (like simple text comparison, running the code a few times, or even advanced AI chatbots) made mistakes. They either missed bugs or thought broken code was fine. ScratchLens was the only one that got it right every time.

Summary Analogy

Imagine two chefs submitting a cake recipe.

Old Tools: Look at the list of ingredients. If Chef A says "sugar" and Chef B says "saccharin," they say "Different!" Or, they taste one bite and say "Same!" even if the cake collapses later.
ScratchLens: It reads the recipe, understands that "sugar" and "saccharin" might be the same function in this context, but also checks if the baking time is different. It tells you: "These cakes will taste the same (Final State), but Chef B's cake will rise faster (Frame/Event)." If it's unsure, it says, "I need to bake a test cake to be sure," rather than guessing.

The paper concludes that for teaching coding, we need tools that understand context (the lens) and causality (the map), rather than just comparing text. ScratchLens is the first tool to do this perfectly for Scratch programs.