Generate in the Chart, Not on the Boundary: Function-Symbol Grounding for Hard Constraints in LTN-GANs

In the world of artificial intelligence, machines have become remarkably good at mimicking the patterns of the real world. They can generate fake but realistic-looking photos of faces, invent plausible-sounding news articles, or create synthetic records of flight schedules and chemical reactions. These tools are invaluable for scientists and engineers who need more data to test theories or train other systems, especially when real data is scarce or too sensitive to share. However, there is a catch. Just because a machine creates something that looks statistically similar to the real thing does not mean it follows the fundamental rules that govern reality. A synthetic flight record might show a plane arriving before it took off, or a chemical molecule might be generated with an impossible energy balance. These errors violate the hard laws of physics and logic that the real data obeys. The challenge for researchers is to build generators that not only look realistic but also strictly adhere to these unbreakable rules, producing data that is both believable and logically sound.

For years, the standard approach to fixing these logical errors has been to treat them as soft suggestions. In this method, the computer is given a penalty whenever it breaks a rule, encouraging it to do better over time. While this helps, it never guarantees perfection; the machine can still occasionally slip up, trading a rule violation for a slightly better-looking image. A more recent and stricter method involves a "constraint layer," a mechanical gate that sits at the end of the generation process. If the machine produces a sample that breaks a rule, this gate physically forces the numbers to change just enough to make the sample valid. It works like a safety net, ensuring that every single output is technically correct. But researchers Nijesh Upreti and Vaishak Belle from the University of Edinburgh discovered a hidden flaw in this safety net. They found that while the gate guarantees the sample is valid, it often destroys the subtle, natural variations that make the data realistic.

The problem lies in how these rules are applied. Many scientific rules are inequalities, such as "the temperature must be greater than zero" or "the drop-off time must be after the pick-up time." In the real world, these gaps are not just zero or one; they have a natural spread. A flight might be delayed by a few minutes, or a chemical reaction might have a small energy buffer. When the standard safety gate fixes a violation, it pushes the sample right up against the edge of the allowed zone, making the gap exactly zero. It does this for every single sample that was even slightly wrong. The result is a dataset where every entry is valid, but the natural distribution of those gaps is gone. The data looks perfect on a checklist, but it feels flat and artificial because the subtle variations that define the real world have been crushed into a single point. Upreti and Belle call this the "collapse" of the margin distribution, and they showed that it happens so quietly that standard tests often miss it, leaving researchers with data that looks good but is fundamentally distorted.

To solve this, the researchers developed a new way of thinking about how the machine builds its samples. Instead of generating a full sample and then checking if it breaks the rules, they changed the blueprint itself. They introduced a method called function-symbol grounding, which acts like a coordinate system inside the allowed zone. Imagine the machine is no longer trying to hit a target and then correcting its aim; instead, it is drawing a map of the entire safe area and picking a spot directly from within it. The machine generates a free, unconstrained number, and then a specific mathematical function translates that number into a value that is guaranteed to be valid. This translation happens by adding a small, positive amount to a base value, ensuring the rule is satisfied by construction. Because the machine is building the sample from the inside out, it never needs to be forced or clamped. The natural variations, the "margins" between the values, remain intact and are learned just like any other feature of the data.

The team tested this approach on four complex, high-resolution datasets involving chemistry, molecular properties, and taxi trip records. In every case, the old method of clamping produced data that was 100 percent valid but had a distorted distribution of the gaps between values. The new method, however, produced data that was also 100 percent valid but perfectly preserved the natural spread of those gaps. The difference was stark: the new method reduced the error in the distribution of these gaps by up to twenty-five times compared to the old method. The researchers also found that they could predict when the old method would fail before even starting the training. They calculated a simple ratio based on the size of the data versus the size of the rule's gap. When this ratio was large, meaning the gap was tiny compared to the data, the old method always collapsed the distribution. The new method worked regardless of this ratio, effectively acting as a universal fix for these types of constraints.

The study also addressed a practical concern: what about rules that involve whole numbers or discrete categories, where the "gap" is not a smooth curve but a single point? The researchers found that their new method works best for smooth, continuous rules, while the old clamping method is actually better for these discrete cases. To get the best of both worlds, they created a hybrid system. Before training begins, the system analyzes each rule to see if it is smooth or discrete, and then automatically chooses the best tool for the job. It uses the new inside-out method for smooth rules and the old clamping method for discrete ones. When tested against the best existing benchmarks, this hybrid system matched or exceeded the performance of the standard clamping method on all datasets, while outperforming it significantly on the difficult, high-resolution cases where the old method failed.

This work changes how we think about generating constrained data. It shows that simply making data valid is not enough; the data must also be realistic in its variations. By shifting from a system that fixes errors after they happen to one that builds validity into the structure of the data itself, the researchers have created a way to generate synthetic data that is not only safe but also faithful to the complex, nuanced reality it is meant to represent. The findings suggest that for any application where the subtle differences between values matter—from designing new molecules to planning logistics—the way we ground our rules in the machine's architecture is just as important as the rules themselves.