WAON: Large-Scale Japanese Image-Text Pair Dataset for Improving Model Performance on Japanese Cultural Tasks

1. Problem Statement

While contrastive pre-training on large-scale image-text datasets has driven significant advances in vision-language models (VLMs), a critical bottleneck exists for Japanese language and cultural adaptation.

• Scarcity of Quality Data: Existing Japanese image-text datasets are either too small (e.g., WIT with ~1M pairs) or suffer from quality issues.
• Limitations of Current Resources:
  • ReLAION (Japanese Subset): Relies on older multilingual CLIP models (mCLIP) for filtering, which are less effective than modern variants, and suffers from insufficient deduplication.
  • Translated Datasets: Datasets created by translating English captions (e.g., ReLAION en→ja) introduce translation noise and inherit Western cultural biases, limiting their utility for tasks requiring deep understanding of Japanese culture.
• Evaluation Gaps: Existing benchmarks for Japanese cultural tasks (e.g., the Recruit dataset) suffer from severe class imbalance (heavily skewed toward food), label-image mismatches, and near-duplicate images, leading to unreliable evaluation metrics.

2. Methodology

A. WAON Dataset Construction

The authors constructed WAON (Web-scale image text Aligned Open Nihongo), a dataset of 155 million Japanese image-text pairs sourced directly from Common Crawl snapshots (2024–2025). The construction pipeline involves:

1. Extraction: Downloading WARC files, extracting Japanese HTML, and identifying (image URL, caption) pairs from alt tags or <figcaption> elements.
2. Pre-filtering: Removing images with invalid URLs or captions lacking Japanese characters (Hiragana, Katakana, CJK).
3. Deduplication (Text/URL): Using a Bloom filter to remove duplicate (image URL, caption) pairs to enhance diversity.
4. Image Download & Quality Filtering:
   • Downloading images using img2dataset.
   • Removing images with extreme aspect ratios, low resolution (<150px), or low color diversity (<32 unique colors) to filter out ads and low-quality graphics.
5. Safety Filtering: Using an OpenCLIP-based NSFW classifier to remove unsafe content (threshold > 0.1).
6. Visual Deduplication: Computing perceptual hashes (pHash) to remove near-duplicate images that share similar visual content despite different URLs.
7. Semantic Alignment Filtering: Using SigLIP2 (a strong multilingual model) to compute cosine similarity between image and text embeddings. Pairs with a score below 0.1 are discarded to ensure high semantic alignment.

B. WAON-Bench Construction

To address evaluation flaws, the authors created WAON-Bench, a manually curated benchmark:

• Scale: 374 classes across 8 categories (animal, building, event, everyday, food, nature, scenery, tradition), totaling 1,870 images (5 images per class).
• Process:
  • Class Definition: Classes were gathered from diverse sources (Wikipedia, Google, field observations).
  • Image Collection: Manual selection of 5 diverse images per class via Google Image Search, prioritizing composition and perspective diversity.
  • Verification: Two annotators verified label-image correspondence, correcting 16 identified errors.
• Improvement: This benchmark offers more than double the class coverage of the previous Recruit dataset and eliminates the severe food-category bias and label mismatches found in prior work.

C. Experimental Setup

• Base Model: siglip2-base-patch16-256 (pretrained on global data).
• Fine-tuning: The model was fine-tuned for 30,000 steps on WAON, ReLAION (Japanese subset), and ReLAION (en→ja translation).
• Evaluation: Zero-shot classification on WAON-Bench, Recruit, and ImageNet (ja); and image-text retrieval on XM3600 (ja).

3. Key Contributions

1. WAON Dataset: The largest publicly available Japanese image-text dataset constructed natively from Japanese web content (155M pairs), surpassing previous resources in scale and quality.
2. WAON-Bench: A rigorous, manually curated benchmark with 374 classes that resolves issues of class imbalance and label noise present in existing Japanese cultural benchmarks.
3. Pipeline Validation: A robust, iterative construction pipeline combining automated filtering (SigLIP, pHash) with manual inspection to ensure high data quality.
4. Open Release: Full release of the dataset (URLs/metadata), fine-tuned models, and code to promote reproducibility.

4. Results

The experiments demonstrate that fine-tuning on WAON yields superior performance compared to existing datasets:

• Japanese Cultural Tasks:
  • WAON-Bench: The WAON-fine-tuned model achieved 95.1% top-1 accuracy, outperforming the ReLAION (ja subset) baseline (93.0%) and the translated ReLAION (76.8%).
  • Recruit Dataset: WAON achieved the best performance among all tested models, significantly outperforming the base SigLIP2 and other fine-tuned variants.
• Global/Western Tasks:
  • ImageNet (ja): WAON improved performance by 1.5 points over the base model. While ReLAION (en→ja) showed a larger gain (3.6 points), this is attributed to the higher proportion of Western imagery in the translated dataset.
  • XM3600 (Retrieval): WAON achieved a massive gain of 35.2 points (38.3% → 73.5%), approaching the performance of the translated dataset (73.8%), proving its effectiveness for cross-lingual retrieval.
• Efficiency: The WAON-trained model converged faster and reached higher accuracy peaks than models trained on the other datasets, validating the "Global Pre-training + Targeted Fine-tuning" paradigm.

5. Significance

• Cultural Adaptation: WAON proves that large-scale, native-language datasets are essential for adapting VLMs to specific cultural contexts, outperforming translated datasets which suffer from cultural bias and translation noise.
• Paradigm Validation: The results support the strategy of pre-training on global data followed by targeted fine-tuning on high-quality, language-specific data as the most effective path for building culturally competent VLMs.
• Research Infrastructure: By releasing WAON and WAON-Bench, the authors provide the community with the necessary tools to develop and rigorously evaluate Japanese-centric vision-language models, addressing a long-standing gap in the field.