In the world of custom cabinetry and built-in furniture, the journey from a sketch to a finished piece begins with a set of drawings. These are not simple pictures; they are complex technical documents that spread the details of a single cabinet across multiple pages, showing it from the front, the side, and in cross-section, often accompanied by handwritten notes and tiny numbers. To turn these drawings into a price quote, a human estimator must read every single part, measure every shelf and panel, and calculate the exact amount of material needed. This reading process is slow, expensive, and prone to error. Recently, researchers have begun asking whether artificial intelligence can do this work. These systems, known as multimodal agents, are computer programs that can both "see" images and "read" text, allowing them to look at a drawing and try to understand what it depicts. The hope has been that these agents could automate the tedious task of counting parts and measuring dimensions, freeing humans to focus on the final price and design. However, early attempts showed that while these machines are good at reading words, they struggle significantly with the visual details of engineering drawings, often missing small numbers or misinterpreting how parts fit together.

A researcher named Donald Dujardin set out to fix this specific problem by building an agent designed to read interior millwork drawings and turn them into a structured list of parts. Instead of simply telling the computer to "do a better job," he treated the agent like a student who needed the right tools and the right information at the right time. Over several months, he tested more than two hundred different changes to see what actually helped the agent get the numbers right. He compared every new idea against a fixed set of correct answers that were prepared before the tests began, ensuring that the results were measured objectively. What he discovered was a clear pattern: the agent improved dramatically when it was given new information it did not have before, but it did not improve at all when it was given more or stricter instructions on how to behave.

The most striking improvement came from changing how the agent looked at the drawings. When the agent tried to read an entire sheet of paper at once, it missed almost all of the tiny dimension numbers, finding only a small fraction of them. This happened because the numbers were so small on the full page that they were nearly invisible to the computer's "eye." The researcher changed the approach by showing the agent the drawing in a grid of smaller, zoomed-in tiles, much like looking at a map through a magnifying glass. This simple change allowed the agent to see the fine details clearly. On one sheet, the number of dimensions found jumped from just four out of twenty-six to twenty-five out of twenty-six. On another sheet with even smaller numbers, the agent went from finding two to finding thirty-four out of sixty-two. The key was not telling the agent to look harder, but simply giving it a view where the details were actually visible.

Another major breakthrough involved how the agent connected the text on the page to the shapes it was seeing. Drawings often have labels like "oak" or "19mm" written near a part, but the computer struggled to know which part those words belonged to. The researcher found that using the text labels from the digital file as fixed reference points, or anchors, solved this problem. When the agent was told to attach its findings to these specific text anchors, its ability to correctly identify which part was which soared from less than half to more than ninety percent. In contrast, when the researcher used fake labels that looked the same but had no real meaning, the agent performed poorly. This proved that the agent needed a concrete, fixed point to hold onto, not just a suggestion to pay attention.

The study also tested the power of examples versus rules. The researcher tried giving the agent a single, clear example of how to use a tool, such as how to ask for a specific measurement, and found that this reduced the number of broken or confused requests by nearly three-quarters. However, when he tried to replace these examples with a long list of written rules telling the agent exactly how to read the drawing, the results were no better than a random string of text of the same length. This was a surprising finding: it suggested that simply writing more instructions or being more firm about what the agent should do did not help. The agent did not need to be told to be more careful; it needed to be shown what a correct action looked like.

Finally, the researcher discovered that the timing of information mattered. When the agent was about to record a measurement, providing it with a specific fact about how cabinets are built in that particular workshop improved the accuracy of the depth measurements. Similarly, when a computer program checked the agent's work and found a mistake, asking the agent a targeted question about that specific error helped it fix the mistake in about one-third of the cases. Without that specific question, the agent rarely corrected its own errors. The study concludes that for these kinds of tasks, the path to better performance is not through better instructions, but through better information. The agent succeeds when it can see the details clearly, has a fixed point to anchor its thoughts, sees examples of correct behavior, and receives specific facts or questions at the exact moment they are needed. This approach offers a practical guide for anyone trying to build intelligent systems that need to understand complex visual documents: give the machine the data it lacks, rather than just telling it what to do.