💻 computer science

AdvTiles: Physical Adversarial Camouflage Clothing against Person Detectors via Learnable Tiles

The paper proposes AdvTiles, a physical adversarial camouflage framework that utilizes learnable tiles with differentiable selection and 3D Gaussian Splatting rendering to achieve high attack success rates against person detectors while maintaining visual naturalness in real-world scenarios.

Original authors: Jinlei Wang, Jiahuan Long, Mingkai Sun, Yafei Guo, Yuanhao Huang, Ming Wang, Junqi Wu, Jiacheng Hou, Hongbo Chen, Xingxing Wei, Tingsong Jiang, Wen Yao

Published 2026-08-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jinlei Wang, Jiahuan Long, Mingkai Sun, Yafei Guo, Yuanhao Huang, Ming Wang, Junqi Wu, Jiacheng Hou, Hongbo Chen, Xingxing Wei, Tingsong Jiang, Wen Yao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking through a busy city, and everywhere you look, there are cameras watching. These aren't just ordinary cameras; they are powered by "person detectors," a special kind of computer brain trained to spot humans instantly. Think of these detectors as hyper-vigilant security guards that never blink, scanning every face and body shape to ensure safety. But here's the twist: these digital guards can be tricked. Just like a magician can fool your eyes with a sleight of hand, researchers have found ways to create "adversarial" patterns—visual illusions that confuse the computer brain, making it see a person as nothing at all. This field of study is like a high-stakes game of hide-and-seek between human ingenuity and machine vision, exploring how to stay invisible to the digital eye without actually disappearing.

For a long time, the best way to hide from these cameras was to wear a giant, weird patch on your shirt, like a bright, chaotic sticker that screamed "I am not a person!" to the computer. While this worked, it was obvious to human eyes and didn't work well if you turned your head or walked away. Then, scientists tried to cover your whole outfit in a strange, natural-looking camouflage, like a chameleon's skin. But even these full-body suits had a problem: they were designed as one giant, unchangeable picture. If the computer saw you from a different angle, the whole picture could get messed up, and the disguise would fail. It was like trying to paint a perfect mural on a piece of fabric that keeps getting wrinkled and stretched; you couldn't fix just one small part without ruining the whole thing.

Enter AdvTiles, a new approach that changes the game by treating your clothes like a giant, customizable puzzle. Instead of painting one big, rigid picture, the researchers built a camouflage suit out of hundreds of tiny, learnable "tiles"—think of them like digital Lego bricks or Scrabble tiles. Each tile has its own unique pattern, and the computer learns how to arrange them perfectly to confuse the detector. The magic trick here is that the computer can swap out individual tiles and rearrange them on the fly, just like a master puzzle solver, to make sure the disguise looks natural to humans but looks like static noise to the camera.

The paper shows that this "tile-based" method is a massive upgrade. By using a clever mathematical tool called a "Straight-through Gumbel-Softmax estimator" (which is basically a way to let the computer make "hard" choices about which tile goes where while still being able to learn from its mistakes), the team could optimize both the patterns on the tiles and where they sit on your shirt. They didn't just stop at a flat picture, though. To make sure the suit worked in the real world, they used a technique called 3D Gaussian Splatting. Imagine taking a 3D model of a person, wrapping it in this tile-patterned fabric, and then spinning it around in a virtual room with changing lights, rain, and different backgrounds. The computer learns to adjust the tiles so that no matter how the camera zooms in, moves around, or changes the lighting, the person remains invisible to the detector.

The results are quite impressive. In digital tests, this new method managed to hide people from person detectors with an Attack Success Rate (ASR) of 86.2% on average, beating out all the previous best methods. When they tested it on specific cameras like YOLOv5, the success rate jumped to 97.5%, meaning the detector almost completely failed to see the person. Even more importantly, when the researchers actually printed these patterns onto real shirts and trousers and wore them in the real world, the disguise held up. Whether the person was standing close to the camera or far away, or whether the camera was looking from the side or straight on, the AdvTiles clothing kept the person hidden. The paper suggests that this approach is a significant step forward, proving that you can have a camouflage that looks like a cool, natural outfit to us, but looks like a confusing glitch to the machine, effectively turning the tables on the digital security guards.

Technical Summary: AdvTiles

Problem Statement

Physical adversarial attacks against person detectors have evolved from localized patches to full-body textures. However, a significant challenge remains in achieving both visual naturalness and strong attack effectiveness simultaneously. Existing natural-looking methods, such as those utilizing Voronoi diagrams parameterized by learnable control points (e.g., AdvCaT), typically optimize camouflage textures as a whole. This holistic approach limits the flexibility to refine local adversarial patterns and their spatial arrangements, making it difficult to maintain effective attacks across varying viewpoints, body poses, and physical conditions while preserving a natural appearance.

Methodology: AdvTiles

The authors propose AdvTiles, a physical adversarial camouflage framework built from learnable tiles. Instead of optimizing a single global texture, AdvTiles represents the adversarial pattern as a layout of discrete tiles, enabling fine-grained control over texture generation. The framework consists of three key modules:

1. Differentiable Texture Generation

To overcome the limitations of global optimization, AdvTiles decomposes the clothing texture into an arrangement of learnable tiles.

Tile Generation: Each tile is defined by a set of learnable parameters controlling color regions, boundaries, and transformations (translation, sharpness, contour frequency, and region size). Adjacent tiles must have matching "keys" along shared edges to ensure seamless composition.
Tile Layout Composition: The framework learns a tile-assignment logit tensor to determine which tile is placed in each grid cell of the clothing UV map. Since tile selection is a discrete process, the authors employ a Straight-through (ST) Gumbel-Softmax estimator. This allows for differentiable tile selection, enabling the joint optimization of tile patterns and their spatial layouts via backpropagation.

2. 3D Gaussian Splatting (3DGS) Based Physical Rendering

To ensure robustness in diverse physical conditions, the method moves beyond 2D texture optimization.

3D Modeling: The generated UV textures are mapped onto the clothing regions of a 3D Gaussian Splatting (3DGS) human model. The geometry and non-clothing appearance of the model are frozen, while only the tile parameters are optimized.
Physical Simulation: The system renders the textured model under sampled viewpoints and scales. It incorporates HDR environment relighting and composites the rendered human with diverse backgrounds (simulating weather conditions like rain, night, and dusk) using structural priors extracted via IC-Light. This process simulates real-world observations, allowing gradients to propagate from the detector back to the tile parameters.

3. Adversarial Losses

The optimization objective combines four loss terms to balance attack success, visual quality, and physical feasibility:

Detection Loss ( $L_{det}$ ): Minimizes the confidence score of the person-class bounding box with the highest Intersection over Union (IoU) with the ground truth.
Diversity Loss ( $L_{div}$ ): Encourages the use of a diverse set of tiles to prevent the model from relying on a few repetitive patterns.
Edge Compatibility Loss ( $L_{edge}$ ): Ensures that adjacent tiles have matching boundary patterns based on predefined compatibility scores.
Seam Loss ( $L_{seam}$ ): Enforces color continuity between adjacent pixels across tile boundaries to avoid visible seams.

Key Contributions

A Tile-Based Framework: The proposal of AdvTiles, which represents adversarial textures as arrangements of learnable tiles, achieving strong attack effectiveness while preserving a natural camouflage appearance.
Differentiable Joint Optimization: The introduction of ST Gumbel-Softmax for jointly learning tile patterns and spatial layouts, providing fine-grained control over texture generation.
Robust Physical Rendering: The leverage of 3D Gaussian Splatting to enhance camouflage robustness under diverse physical conditions, including variations in viewpoints, scales, illuminations, and backgrounds.
Comprehensive Validation: Extensive experiments demonstrating superior performance over state-of-the-art (SOTA) methods across multiple detectors and in real-world physical scenarios.

Experimental Results

The authors evaluated AdvTiles against various baselines, including patch-based attacks (AdvPatch, NatPatch) and texture-based attacks (AdvTexture, AdvCaT), across multiple detector architectures (YOLO series, RetinaNet, FCOS, SSD, DETR variants, Faster R-CNN).

Digital Performance: AdvTiles achieved an average Attack Success Rate (ASR) of 86.2% across multiple detectors, outperforming existing SOTA methods. In specific comparisons against YOLOv5, AdvTiles reached an ASR of 97.5%, with a recall of 2.5% and an F1-score of 4.5%, significantly outperforming AdvCaT (73.6% ASR).
Black-Box Transferability: The method demonstrated strong transferability, maintaining high attack effectiveness when applied to detectors other than the one used for training (e.g., textures generated for YOLOv8 performed well on YOLOv5 and FCOS).
Physical Robustness:
- Distance and Angles: AdvTiles maintained effective performance across distances from 3.0m to 9.0m and horizontal viewing angles from 0° to 270°, with ASRs consistently above 95% for most angles.
- Real-World Fabrication: The optimized camouflage was fabricated into wearable clothing. Physical experiments in indoor and outdoor environments confirmed that AdvTiles consistently evaded person detection across diverse distances, angles, and backgrounds, outperforming both AdvCaT and ordinary camouflage.

Significance and Claims

The paper claims that AdvTiles advances the field of adversarial camouflage by addressing the trade-off between visual naturalness and attack robustness. By moving from holistic texture optimization to a learnable tile-based approach, the method achieves finer-grained control over adversarial patterns. The integration of 3D Gaussian Splatting for physical rendering ensures that the generated camouflage is robust to real-world variations in lighting, viewpoint, and background.

The authors state that this work takes a step toward the practical deployment of adversarial attacks in the real world, demonstrating that it is possible to create adversarial clothing that is both effective against person detectors and visually natural. The successful fabrication and testing of the clothing in real-world scenarios validate the framework's potential for physical adversarial attacks.