💻 computer science

Diamojism: Semantic-Perceptual Tile Selection for Large-Format Emoji Mosaic Rendering

This paper introduces Diamojism, a novel photomosaic rendering architecture that enhances large-format emoji mosaics by integrating semantic theme scoring and iterative palette correction into a diamond tile grid system, thereby improving thematic relevance and color accuracy without relying on generative models.

Original authors: Michael Jacobs

Published 2026-07-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Michael Jacobs

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, famous painting—maybe something by Van Gogh or da Vinci—and you want to recreate it, but you can't use paint. Instead, you have to build the whole image using thousands of tiny, square stickers. But these aren't just any stickers; they are emojis. 🌊🌙🎨

This is the world of Diamojism, a new way of making art that turns a classic painting into a mosaic made entirely of emojis. But here's the twist: the computer program that does the work isn't just looking at colors. It's also reading the meaning of the emojis.

The Old Way vs. The New Way

For a long time, computer programs that make these mosaics acted like a very strict, color-blind librarian. If you asked for a blue patch of sky, the librarian would just grab the bluest sticker available, even if it was a blue teardrop or a blue geometric shape. They didn't care what the sticker was, only what it looked like.

This paper argues that's a bit boring. The authors, led by artist Michael Jacobs, built a smarter system. They treated their library of 3,699 emojis not just as a pile of pictures, but as a library of ideas.

The "Smart Librarian" System

The new system works like a two-step game to pick the perfect emoji for every tiny square of the painting:

The Look-Check (Perception): First, the computer asks, "Does this emoji look like the color and texture of this spot in the painting?" It checks the shade of blue, the amount of "fuzziness" (edges), and where the lines are. This is the most important part. If an emoji doesn't look right, it's out.
The Vibe-Check (Semantics): Once the computer has a shortlist of emojis that look right, it asks a second question: "Does this emoji fit the theme?"
- If the painting is The Great Wave, and the computer is picking an emoji for the water, it prefers emojis tagged with "ocean," "wave," or "fish" over a blue teardrop or a blue square, even if the teardrop is the exact same shade of blue.
- If the painting is Starry Night, it prefers "moon" or "star" emojis for the sky.

The paper shows that this "vibe-check" doesn't ruin the picture. In fact, it makes the art make more sense without changing how the colors look. For The Great Wave, this trick increased the number of "on-theme" emojis by 13.6 percentage points. That means the water actually looked like water, not just blue stuff.

The "Diamond" Puzzle

The painting isn't just a grid of squares. The authors rotated the whole grid by 45 degrees, so the tiles are actually diamonds. This helps the emojis flow better with the brushstrokes in the original paintings.

To make sure the details pop, the system uses a "three-geometry" map. Imagine the artist drawing on the painting with three different colored markers:

A Box: They draw a box around a face or a flower. The computer knows to use tiny, detailed emojis there.
A Line: They draw a curvy line along a wave or a tree branch. The computer follows that curve, using small emojis right along the line.
A Painted Mask: They paint a whole area (like the sky) to tell the computer, "Make this whole zone a bit more detailed."

This lets the computer zoom in on the eyes of a portrait or the swirls of a storm, while keeping the background simple, just like a real artist would.

The "Color Fix" Loop

Sometimes, the emoji library is a bit weird. Maybe the emojis are all too bright or too yellow compared to the original painting. The authors built a "color fix" tool that learns from its own mistakes.

Here's how it works:

The computer makes a first draft of the mosaic.
It looks at the result and says, "Oops, the blue emojis are too bright."
It adjusts the "memory" of the blue emojis to be a little darker.
It makes the mosaic again.

This isn't magic; it's a feedback loop. For one painting, Birth of Emoji, this fix reduced the color errors by 40% for lightness and 69% for color intensity. However, the paper notes this doesn't work for every painting. For Starry Night, the stars were so bright and the sky so dark that fixing one made the other worse. So, the authors had to turn the fix off for that one.

What This Paper Says (and Doesn't Say)

The paper is very careful about what it claims. It does not say this system uses AI to create new emojis or to "think" like a human artist. The emojis are pre-made; the computer just picks the best ones. It does not say this is the perfect way to make art; it says it's a tool that works well for specific, large-scale prints (ranging from 24 to 30 inches).

The authors measured their results on five specific paintings. They found that adding the "theme" check made the art more interesting without making it look blurry or wrong. They also found that their system is fast enough to handle millions of pixels, producing prints with up to 33,358 individual emoji tiles.

The Big Picture

The main idea is simple: Meaning matters. By teaching the computer to care about what an emoji means (a wave vs. a teardrop) in addition to what it looks like, you can create mosaics that feel more alive. It's like building a castle out of LEGO bricks: you could use any blue brick for the sky, but if you use the specific "cloud" or "sun" bricks, the castle feels more real.

This isn't a magic wand that solves every art problem, but it's a clever new way to mix old masterpieces with modern symbols, proving that even a simple emoji can carry a lot of weight when placed just right.

Technical Summary: Diamojism: Semantic-Perceptual Tile Selection for Large-Format Emoji Mosaic Rendering

Problem Statement
Traditional photomosaic rendering systems treat tile libraries as purely visual datasets, selecting candidates based solely on color and edge similarity (nearest-neighbor in color space). This approach ignores the semantic content of the tiles. Furthermore, existing systems often struggle with tile diversity (repeating the same image) and systematic color mismatches between the library's natural distribution and the source image's palette, particularly in large-format, high-resolution outputs. The paper addresses the need for a rendering architecture that integrates semantic meaning into the selection process without sacrificing perceptual fidelity, while also managing the computational constraints of large-scale production (up to 33,000 tiles at 512 DPI).

Methodology
The proposed system, "Diamojism," operates on a 45°-rotated diamond tile grid and utilizes a five-signal scoring model over an adaptively subdivided grid. The pipeline consists of five phases:

Preprocessing (Phase 0): The Google Noto Emoji set (3,699 base glyphs) is rasterized at five resolution tiers (32–512 px) with four cardinal rotations, creating a feature database of ~74,000 records. Features include CIE Lab* mean color, a 3×3 spatial edge vector (capturing edge distribution), and semantic tags derived from Unicode subgroup taxonomy and Emojibase keywords.
Importance Map Construction (Phase 3): A unified spatial importance field is generated by combining automated maps (face, eye, mouth detection) with a novel three-geometry artist region system. Artists can define regions using bounding boxes, polylines (for curves like brushstrokes), or painted masks. These regions drive adaptive subdivision with independent minimum tile size constraints per region.
Tile Plan Construction (Phase 4): A recursive three-gate subdivision algorithm determines tile density. Gates evaluate detail ratio, target size (driven by importance maps), and hard limits (min size/depth). This allows fine subdivision in specific areas (e.g., eyes or wave crests) while keeping background areas coarse.
Candidate Selection and Scoring (Phase 5): For each leaf tile, a candidate pool is retrieved via KD-tree query. Selection uses a two-pass scoring model:
- Pass 1 (Perceptual): Calculates a weighted sum of color (LAB MSE), edge density, and spatial edge distribution scores.
- Pass 2 (Adjustments): Applies a usage diversity penalty (to prevent repetition) and an optional semantic theme scoring layer. The theme layer acts as a multiplicative modifier, biasing selection toward emoji whose semantic tags match the source image's theme (e.g., "marine" for a wave painting) without overriding perceptual quality.
Iterative Palette Correction: An optional feedback mechanism uses placement logs from a completed render to fit per-emoji, per-channel CIE Lab* scale-and-shift corrections via least-squares regression. These corrections adjust the in-memory feature database for subsequent renders, addressing systematic color gaps before selection occurs.

Key Contributions
The paper identifies four primary architectural contributions:

Three-Geometry Artist Region System: A unified spatial importance field derived from bounding boxes, polylines, and painted masks, enabling independent minimum tile size constraints for different regions within a single render.
Five-Signal Two-Pass Scoring Model: A candidate selection architecture that separates perceptual scoring (color, edge, spatial fidelity) from diversity penalties and semantic theme bias. This ensures semantic alignment acts as a secondary bias rather than a primary filter.
Semantic Theme Scoring Layer: A mechanism that integrates Unicode subgroup taxonomy and keyword annotations as a multiplicative factor in the scoring loop, allowing the system to align with the source image's thematic vocabulary.
Iterative Palette Correction Mechanism: A render-history feedback loop that corrects systematic candidate routing errors by adjusting the feature database's color values, rather than post-processing pixel values after placement.

Results
The architecture was evaluated across five large-format production works derived from paintings by da Vinci, Hokusai, Seurat, Van Gogh, and Botticelli.

Semantic Theme Scoring: In controlled comparisons across four works with configured theme presets, enabling semantic scoring increased the on-theme placement rate by 6.6 to 13.6 percentage points (e.g., +13.6 pp for The Great Wave of Emoji). Crucially, this increase occurred with negligible change in placement entropy (Shannon entropy remained stable), indicating the system biased the selection field without concentrating placements in a narrow subset. Mean color fidelity ( $\Delta E$ ) remained within measurement noise.
Palette Correction: In two works (Emoji Lisa and Birth of Emoji) where the luminance bias was consistent across tile sizes, palette correction reduced systematic lightness residuals by 40% and chroma residuals by 69%. In three other works, the mechanism was rolled back because the bias varied by tile size (e.g., small detail tiles vs. large background tiles), making a single per-emoji correction insufficient.
Performance: The system successfully rendered 5,508 to 33,358 tiles with zero fallback selections (all tiles found candidates via the primary indexed path), demonstrating the robustness of the layered indexing structure.

Significance
The paper claims that treating a tile library as a semantically annotated dataset, rather than a purely visual one, expands the expressive capacity of photomosaic rendering without requiring generative models or post-hoc style transfer. The system demonstrates that semantic alignment can be integrated into the selection process as a secondary bias, allowing the artist to control the degree of thematic alignment while maintaining perceptual fidelity. The architecture is presented not as a benchmark-winning algorithm, but as a production tool built to meet specific artistic requirements for large-format fine art prints, bridging the gap between computational rendering and artistic intent.