Structure-guided fusion of frozen visual representations for small-sample Bai furniture pattern recognition

In the quiet corners of museums and family archives, thousands of photographs and sketches wait to tell the story of the Bai people, an ethnic group in southwestern China known for their intricate wooden furniture. These pieces are not merely functional; they are covered in decorative patterns that serve as visual records of local craft knowledge, aesthetic choices, and cultural history. The designs range from geometric grids and stylized animals to plants, ancient vessels, and Chinese characters, often carved or painted directly onto cabinets, chairs, and beds. For decades, organizing this visual heritage has relied on human experts manually sorting, describing, and interpreting these images. This process is slow, expensive, and difficult to scale as collections grow, making it hard to compare patterns across different institutions or to retrieve specific designs efficiently.

The challenge for computers is that these patterns are subtle and varied. A plant stem might look like an animal's outline, or a carved border might mimic the strokes of a written character. Furthermore, the images themselves are inconsistent; some are clean line drawings, while others are photographs of weathered wood with uneven lighting and texture. Modern artificial intelligence has made great strides in recognizing images, but these systems usually require massive amounts of labeled data to learn, a resource that simply does not exist for niche cultural collections like the Bai furniture patterns. When researchers try to apply standard, powerful computer vision models to such small datasets, the systems often fail to distinguish between these fine details, or they lose the specific local textures that define the art. The question became whether a computer could learn to recognize these specific cultural motifs without needing to be retrained from scratch, and if it could do so by paying attention to the very lines and textures that make the patterns unique.

A team of researchers set out to solve this by building a system that respects the limits of the available data while leveraging the power of existing technology. They created a new dataset called BaiFDP, which contains 620 images of Bai furniture patterns sorted into six main categories: geometric, animal, antique objects, plants, Chinese characters, and a difficult "combined" category where multiple types of motifs appear together. Instead of trying to teach a computer everything from the beginning, they started with a massive, pre-trained visual model that had already learned to see the world. However, they made a crucial decision to stop the computer from using the part of its brain that translates images into language. They found that the final output of such models, designed to match pictures with words, actually smoothed over the fine details needed to tell a geometric grid from a stylized leaf. Instead, they looked deeper into the model's internal layers and discovered that a specific layer near the end of the visual processing chain held the sharpest, most detailed spatial information about the image.

To bridge the gap between this powerful but general visual memory and the specific needs of Bai patterns, the researchers added a second layer of analysis based on old-fashioned, handcrafted rules. They taught the system to explicitly measure the direction of edges, the local patterns of light and dark, and the statistical relationships between neighboring pixels. These measurements act like a set of precise rulers and protractors, capturing the exact orientation of a carved stroke or the texture of a painted surface. The innovation lay in how they combined these two sources of information. Rather than simply stacking the general visual data and the specific texture data together, they designed a system where the texture measurements could actively guide the visual data. One approach allowed the texture rules to act as a gate, deciding how much weight to give to specific parts of the image. A more advanced approach allowed the system to look at a specific visual detail and ask, "What does the texture data say about this exact spot, and what does it say about other spots nearby?" This allowed the computer to understand that a specific curve might be part of a larger animal shape, even if the curve itself looked like a plant stem.

The results showed that this method worked better than relying on the visual model alone or simply combining the data without interaction. The system that used the guided attention mechanism correctly identified the patterns in 91.83 percent of the test images, an improvement over the 85.81 percent accuracy of the visual model by itself. However, the researchers noted that while this higher mean represented a practically relevant improvement, the statistical analysis did not establish definitive superiority; the confidence interval for the difference crossed zero, and the adjusted statistical test was not significant. The researchers also found that the system was good at knowing when it was unsure. When the computer was confident, it was almost always right, but when the patterns were mixed or ambiguous, the system flagged these cases with lower confidence scores. This is vital for cultural heritage work, where a wrong guess could lead to a misinterpretation of history. To demonstrate the practical value, the team built a tool that takes the computer's prediction and uses it to retrieve verified records from a curated database. If the system identifies a pattern as a "combined" motif, it does not guess the meaning; instead, it presents the user with the most likely options and the specific records that support them, ensuring that any cultural explanation is grounded in evidence rather than the computer's imagination.

The study confirms that for small, specialized collections, the best approach is not to force a massive model to learn everything from scratch, but to guide its existing knowledge with explicit, measurable details about the object's structure. The researchers found that the internal layers of a frozen model, when paired with local texture analysis, could capture the nuance of hand-carved art that broader models miss. While the statistical difference between their best method and a slightly simpler version was not overwhelmingly large, the trend was clear: allowing the system to dynamically focus on structural evidence improved its ability to distinguish between complex, overlapping motifs. The work does not claim that the computer now understands Bai culture or history; rather, it provides a reliable, data-efficient way to organize and index these visual resources, leaving the deep cultural interpretation to human experts who can use the computer's accurate sorting to find the right records. This creates a partnership where the machine handles the heavy lifting of visual organization, and the human provides the context, ensuring that the digital preservation of these cultural treasures remains both accurate and respectful.