Patch-Based Spatial Authorship Attribution in Human-Robot Collaborative Paintings

Imagine you are looking at a beautiful, abstract painting. You know for a fact that two artists created it together: one is a human, and the other is a robot. They didn't take turns; they painted on the same canvas at the same time, mixing their brushstrokes like a dance.

Now, imagine you are a detective trying to figure out: "Who painted this specific spot? Was it the human or the robot?"

This is exactly the problem Eric Chen and Patricia Alves-Oliveira solved in their new paper. They built a digital "forensic microscope" that can look at a painting, zoom in on tiny squares, and tell you who likely made the mark.

Here is the story of how they did it, explained simply:

1. The Problem: The "Who Did What?" Mystery

In the past, art experts (connoisseurs) could tell if a painting was by Van Gogh or Picasso just by looking at the brushstrokes. But today, robots are learning to paint. They can make strokes that look surprisingly human.

When a human and a robot paint together, the lines get messy. It's hard to say, "This part is the human, and that part is the robot." The old ways of checking art (like looking at the whole picture or using expensive, high-tech scanners) don't work well here because:

We don't have thousands of paintings to study (robots haven't been painting long enough).
We need to know where the robot painted, not just if the robot painted.

2. The Solution: The "Puzzle Piece" Approach

Instead of looking at the whole painting at once, the researchers chopped the images into thousands of tiny 300x300 pixel squares (like cutting a giant jigsaw puzzle into tiny pieces).

They trained a computer brain (an AI) to look at just one tiny square at a time and ask:

"Is this empty canvas?"
"Is this a human brushstroke?"
"Is this a robot brushstroke?"

The Analogy: Think of it like a taste tester at a soup factory. Instead of tasting the whole pot, they take a tiny spoonful. Even if the soup is a mix of two chefs' recipes, a trained palate can sometimes tell, "Hmm, this spoonful tastes more like Chef A's spice blend."

3. The Training: Learning from a Small Group

Usually, AI needs to eat thousands of examples to learn. But here, they only had 15 paintings (7 by a human, 8 by a robot). That's like trying to learn to recognize a friend's handwriting when you've only seen 15 notes.

To make this work, they used a clever trick called "Leave-One-Out" testing:

They trained the AI on 14 paintings.
They tested it on the 15th painting (which it had never seen).
Then they swapped them around and did it again.

This proved the AI wasn't just memorizing the specific paintings; it actually learned the style of the human vs. the style of the robot.

4. The Results: The AI Got It Right!

The results were impressive:

Accuracy: The AI correctly identified the author of a tiny square 88.8% of the time.
Beating the Competition: It did much better than older methods that just looked at texture patterns or used pre-trained models (which are like general knowledge books rather than specific art experts).

5. The "Uncertainty" Superpower: Finding the Gray Areas

This is the most creative part of the paper. What happens when the human and robot paint exactly on top of each other? The AI shouldn't just guess; it should say, "I'm confused."

The researchers taught the AI to measure its own confidence (or "uncertainty").

Low Uncertainty: "I'm 99% sure this is the human."
High Uncertainty: "I see features of both the human and the robot here. I'm not sure."

The Metaphor: Imagine a security guard at a club.

If he sees someone in a clear red shirt, he says, "Human."
If he sees someone in a clear blue shirt, he says, "Robot."
But if he sees someone wearing a purple shirt (a mix of red and blue), he doesn't guess. He raises his hand and says, "This is a collaboration zone."

The study found that in the "mixed" paintings, the AI was significantly more confused (64% more uncertain) than in pure paintings. This proved the AI wasn't failing; it was successfully detecting the "purple shirt" moments where the human and robot styles blended.

6. Why Does This Matter?

This isn't just about robots and art. It's about trust and ownership in a world where AI is creating things.

For Artists: It helps prove who contributed what to a collaborative piece.
For Collectors: It helps verify if a painting is truly a human-robot collaboration or just a robot pretending to be human.
For the Future: It shows we can solve complex mysteries with very little data and simple tools (like a regular flatbed scanner), without needing million-dollar labs.

In a nutshell: The researchers built a smart digital detective that can zoom in on a painting, identify the "fingerprint" of a human hand versus a robot arm, and even point out the exact spots where they worked together. It's a new way to tell the story of who made art, even when the story is a team effort.

1. Problem Statement

As AI and robotic systems increasingly participate in creative production, determining authorship in collaborative artworks has become a critical challenge for art markets, legal contexts, and artists.

The Core Challenge: Traditional art authentication assumes a single author. However, in human-robot collaborative paintings, authorship is spatially varying (i.e., different parts of the canvas are created by different agents).
Limitations of Existing Methods:
- Large-scale models: Require massive datasets (thousands of works per artist), which do not exist for contemporary creators or robots with limited output.
- Scientific imaging: Relies on specialized, expensive hardware (e.g., 3D surface topography, X-ray) to detect physical cues.
- Digital-only detection: Current AI detection methods focus on digital artifacts (neural rendering, compression) and do not apply to physical paintings digitized via scanning.
Goal: Develop a method to attribute authorship at the patch level (identifying where the human or robot painted) using only commodity hardware and small datasets, while quantifying uncertainty in ambiguous collaborative regions.

2. Methodology

A. Dataset Construction

Data Source: 15 physical abstract paintings created with acrylics on canvas.
- 7 Human-only paintings: Created by a single artist based on the prompt "air."
- 8 Robot-only paintings: Generated by a 6-DOF robotic arm (UF850) using the CoFRIDA framework (image-to-action pipeline).
- 5 Hybrid paintings: Created collaboratively where the robot and human painted simultaneously on the same canvas.
Digitization: All paintings were scanned using commodity flatbed scanners at 1200 DPI to ensure consistent lighting and resolution.
Patch Extraction: Images were divided into 300×300 pixel patches with 50% overlap (stride 150), yielding ~137,000 patches.
Labels: Patches were labeled as Blank, Human, or Robot. Note: For hybrid paintings, ground truth is inherently ambiguous; manual annotation was used only to identify "hybrid regions" for uncertainty analysis, not for training.

B. Model Architecture

Framework: A custom, compact VGG-style Convolutional Neural Network (CNN) adapted from the PigeoNET framework.
Structure:
- Input: Grayscale 300×300 patches.
- Layers: Five convolutional blocks (filters: 32→64→128→256→512), each with two 3×3 conv layers, batch normalization, ReLU, and 2×2 max pooling.
- Head: Dropout (0.4), Global Average Pooling, and three fully connected layers (512→512→128→3) outputting logits for the three classes.
Rationale: A compact architecture was chosen over fine-tuning large foundation models (like ResNet or DINOv2) to prevent overfitting on the small dataset and to better capture fine-grained, task-specific brushstroke cues.

C. Training Strategy

Validation: Leave-One-Painting-Out (LOPO) Cross-Validation. In each of the 15 folds, one entire painting was held out for testing while the model trained on the remaining 14. This ensures the model generalizes to unseen artworks, not just unseen patches of the same artwork.
Class Imbalance: Addressed using weighted cross-entropy loss. Weights were adjusted to downweight the "Blank" class (trivial) and balance the "Human" vs. "Robot" classes.
Augmentation: Rotation (±15°), flips, random cropping, Gaussian blur, and padding.

D. Uncertainty Quantification for Hybrid Artworks

Since ground truth is ambiguous in collaborative works, the authors used Conditional Shannon Entropy to detect mixed authorship:

Filtering: Only patches with significant paint content ( $p_{human} + p_{robot} > 0.2$ ) were considered.
Renormalization: Probabilities were renormalized to exclude the "Blank" class.
Entropy Calculation: $H = -P_{human} \log_2(P_{human}) - P_{robot} \log_2(P_{robot})$ $H = - P_{h u man} lo g_{2} (P_{h u man}) - P_{r o b o t} lo g_{2} (P_{r o b o t})$ .
- $H=0$ : High confidence (clearly human or robot).
- $H=1$ : Maximal ambiguity (equal probability of both).

Hypothesis: Patches in hybrid paintings will exhibit significantly higher entropy than pure paintings because they contain overlapping stylistic cues from both agents.

3. Key Results

A. Classification Performance (Pure Paintings)

Patch-Level Accuracy: The proposed CNN achieved 88.8% accuracy (86.7% painting-level via majority vote).
Comparison to Baselines:
- LBP + Random Forest: 65.9% (High variance, low robustness).
- ResNet-50 + SVM: 81.96%.
- DINOv2 + SVM: 84.70%.
- Proposed CNN: 88.79%.
Per-Class Recall:
- Robot: 90.24%
- Human: 88.33%
- Blank: 85.01%
Significance: The custom CNN outperformed pretrained models, demonstrating that learning directly from painting patches captures fine-grained brushstroke signatures better than generic natural image features.

B. Hybrid Painting Analysis (Uncertainty)

Entropy Elevation: Hybrid patches showed a 64% higher uncertainty (median entropy 0.18) compared to pure human (0.11) or pure robot (0.13) patches.
Statistical Significance: The difference was highly significant (p=0.003 via Mann-Whitney U test).
High-Uncertainty Regions: 33.1% of hybrid patches had entropy > 0.5, compared to only ~16% in pure paintings.
Interpretation: The model did not fail; rather, the elevated entropy correctly signaled the presence of mixed stylistic cues where definitive attribution is impossible.

C. Validation of Patch-Based Approach

The authors verified that the high accuracy was not due to "pseudoreplication" (overfitting to spatial correlations).
When testing a constrained baseline (sampling only one patch per painting), accuracy dropped to 62.3%, proving that the patch-based approach provides genuine statistical power by treating local regions as independent training samples.

4. Key Contributions

Spatial Attribution Framework: Introduced a method to move beyond "who made this?" to "where and how did contributions differ?" in collaborative art.
Data-Efficient Learning: Demonstrated that high-accuracy attribution is possible with minimal data (15 paintings) and commodity hardware (flatbed scanners), without needing large museum-scale datasets.
Uncertainty as a Signal: Proposed using predictive entropy as a principled metric to identify regions of mixed authorship in collaborative works where ground truth is undefined.
Performance Benchmark: Established a new state-of-the-art for small-scale, human-robot style classification, outperforming both hand-crafted texture descriptors and large pretrained foundation models.

5. Significance and Future Work

Practical Impact: Provides a forensic tool for artists, collectors, and legal systems to document authorship in the emerging field of human-robot collaboration.
Methodological Shift: Moves art authentication away from reliance on expert connoisseurship or expensive scientific imaging toward accessible, computational spatial analysis.
Future Directions:
- Extending the framework to multiple human artists and different robotic systems.
- Investigating cross-system transferability.
- Incorporating temporal data (stroke ordering, robot motion trajectories) to further refine attribution.

Conclusion: The paper successfully demonstrates that a lightweight, patch-based CNN can distinguish human and robotic painting styles with high accuracy in data-scarce environments and effectively use model uncertainty to map the "gray areas" of collaborative creativity.