CLEM: A Behavior-Centric Software Quality Measurement Framework for Structural Change Monitoring

Imagine you are watching a city grow. You could count how many bricks are laid each day, or you could check if the city council followed the rules. But there's a third, more interesting way to look at a city: watch how the buildings change. Do people knock down old walls to add a new room? Do they build a new wing that attaches to the side without touching the main house? Do they just flip a switch to change the lighting? Or do they simply rearrange the furniture? In the world of computer software, this is exactly the question researchers are asking. Software isn't just code; it's a living system that has to change constantly to stay useful. If a system only changes by tearing down its own walls, it eventually becomes a shaky, dangerous mess. But if it changes by adding new wings or flipping switches, it stays strong and flexible. This is the core of "software quality"—not just whether the code works today, but whether it can keep growing without falling apart tomorrow.

This paper introduces a new tool called CLEM (Change Localization and Externalization Measurement) to answer that question. Instead of just counting how much code was changed, CLEM acts like a detective that watches how developers fix or update a system. It sorts every change into one of four "personas":

1. Modification (M): The "Knock-Down" approach. Changing the core code directly. It's fast but risky, like hacking a hole in a wall to add a door.
2. Extension (E): The "Add-On" approach. Building new features that plug into the system without touching the core, like adding a new room to a house.
3. Low-code (L): The "Flow-Chart" approach. Using visual tools or rules to change behavior, like a business manager rearranging a workflow without writing code.
4. Configuration (C): The "Switch" approach. Just changing settings or parameters, like turning a dial to change the volume.

The researchers tested this idea on three different software projects: two public ones from a large tech ecosystem and one private healthcare app. They found that CLEM can clearly tell the difference between a system that is "healthy" (mostly using add-ons and switches) and one that is "sick" (constantly hacking its own core). However, they also discovered something surprising: knowing how a system changes doesn't automatically predict if it will have more bugs next month. It's a great tool for understanding the structure of a system, but it's not a crystal ball for predicting future errors.

The Detective's New Notebook: How CLEM Works

Think of software development like a busy kitchen. For years, chefs (developers) have been measured by how many dishes they cook (activity volume) or how clean the kitchen is at the end of the night (static code checks). But what if the kitchen is falling apart because every time they need a new spice, they have to smash a wall to get to the pantry? That's the problem CLEM solves. It doesn't just count the dishes; it watches the method the chefs use to get ingredients.

The paper proposes that every time a software system is updated, the change happens in one of four ways, and the mix of these ways tells us everything about the system's health.

• Modification (M) is the "Brute Force" method. It's like a chef grabbing a sledgehammer to break a wall because they need a new shelf. It gets the job done fast, but if you do it too much, the whole building becomes unstable.
• Extension (E) is the "Modular" method. It's like building a new, detachable cart that rolls into the kitchen. The chef doesn't touch the walls; they just add a new tool. This is safer and keeps the core structure intact.
• Low-code (L) is the "Blueprint" method. Imagine a manager drawing a new flow on a whiteboard that tells the robots what to do, without the robots needing to be reprogrammed. It's a higher-level way of changing things.
• Configuration (C) is the "Dial" method. It's just turning a knob to make the oven hotter or the lights brighter. No construction needed at all.

The authors argue that a healthy, long-lasting software system should rely more on Extension, Low-code, and Configuration, and less on Modification. If a system is constantly "Modifying" its core, it's likely accumulating "technical debt"—a fancy way of saying it's borrowing stability from the future and will have to pay it back with interest later.

The Experiment: Watching Three Kitchens

To see if this idea works, the researchers went on a field trip to three different "kitchens" (software repositories). They didn't just look at the final dishes; they watched the chefs' hands for months.

1. The "Fit" Kitchen (fit-framework): This was a public project designed to be a plugin system. They expected it to be full of "Extensions" (E).
2. The "App" Kitchen (app-platform): This was another public project, but it was built for low-code visual design. They expected it to be full of "Low-code" (L) and "Configuration" (C).
3. The "Antisuger" Kitchen: This was a private healthcare app for managing blood sugar. It was built by a different team with different tools. They expected it to be in an early, chaotic phase, likely full of "Modifications" (M).

The researchers analyzed 607 specific updates (commits) across these projects. They used a set of transparent rules to look at the files being changed. If a file was in a "plugin" folder, they counted it as Extension. If it was a "flow" file, they counted it as Low-code. If it was a core code file, it was Modification.

What They Found: The Systems Looked Different

The results were exactly what the "healthy kitchen" theory predicted.

• The App-platform was indeed very "externalized." About 69.5% of its changes were Extensions, with very little direct hacking of the core. Its "CLEM-ES" score (a measure of how much change was pushed away from the core) was a strong +0.685.
• The Fit-framework was a mix. It had a lot of Extensions (33.4%), but it also had a significant chunk of Modifications (29.1%). Its score was +0.418, showing it was healthier than a pure mess, but not as "externalized" as the App platform.
• The Antisuger healthcare app was the opposite. It was almost entirely "Modification" dominant, with 83.0% of its changes being direct core edits. Its score was -0.659, indicating it was still in a fragile, "hacking the walls" phase.

This proved that CLEM can successfully spot the difference between a system that is growing by adding wings and one that is growing by smashing walls. The researchers even checked if their rules were fair by having two humans look at 160 random updates. They agreed 100% of the time on the main category, which suggests the rules are solid and reproducible.

The Twist: Structure Doesn't Predict Bugs (Yet)

Here is the part where the paper gets very careful. You might think, "If a system is hacking its own walls (high Modification), it should break more often, right?" The researchers tested this. They looked at whether the CLEM scores could predict if the system would have more "bug fixes" in the following month.

The answer? No clear link.
In their data, the "Modification" score didn't reliably predict if the next month would be full of bug fixes. The "CLEM-ES" score (how externalized the changes were) had almost zero correlation with future bug fixes in this specific sample.

This is a crucial finding. The authors explicitly state that CLEM is not a magic crystal ball for predicting defects. It doesn't replace the old ways of counting bugs or code churn. Instead, it offers a different kind of insight. It tells you about the structural posture of the system. A system with a high Modification score might not have more bugs today, but it is building a structure that is harder to maintain and more likely to become fragile over time. It's like a building that is structurally unsound; it might not collapse today, but the blueprint is bad.

Why This Matters

The paper concludes that CLEM is a powerful new lens for software managers. It moves the conversation from "How much code did we write?" to "How are we changing our system?"

• If you see a team constantly doing Modifications, it's a signal to pause and ask: "Why are we breaking our own walls? Can we build a plugin instead?"
• If you see a team doing mostly Extensions and Configurations, it suggests the system is maturing and becoming more stable.

The authors are honest about the limits of their work. They admit their sample size was small (only a few months of data from three projects) and that the "bug prediction" part didn't work as hoped. They suggest that CLEM is best used as a complementary tool—a way to keep an eye on the structural health of a system alongside traditional metrics. It's not a final verdict on quality, but a very clear, auditable way to see if a software system is learning to grow up or if it's stuck in the habit of breaking its own foundation.

In short, CLEM gives us a vocabulary to talk about the shape of change. It helps us see if our software is building a skyscraper or just stacking bricks on top of a shaky pile, and that distinction might be the most important thing we can measure for the long-term survival of any digital system.