Steering Generative Models for Accessibility: EasyRead Image Generation

1. Problem Statement

EasyRead is an accessibility framework designed to support individuals with intellectual disabilities, cognitive impairments, and low literacy by presenting information through simplified text paired with clear, unambiguous pictograms.

• The Challenge: Producing EasyRead-compliant images at scale is currently bottlenecked by the high cost and expertise required for manual design.
• The Limitation of Current AI: Modern state-of-the-art text-to-image diffusion models (e.g., Stable Diffusion) tend to generate visually rich, detailed, and stylistically complex images. These outputs often violate EasyRead principles such as minimalism, strong contrast, and the unambiguous depiction of key concepts.
• The Gap: There is a lack of systematic, quantitative methods to evaluate "EasyReadness" (clarity and simplicity) in generated images, and no existing pipeline effectively adapts generative models to meet these specific cognitive accessibility constraints.

2. Methodology

The authors propose a unified pipeline to generate EasyRead pictograms by fine-tuning a pre-trained diffusion model on a curated dataset.

A. Data Curation and Augmentation

The training corpus combines three public datasets:

1. OpenMoji: 4,295 vector-based icons.
2. ARASAAC: 11,972 augmentative and alternative communication pictograms.
3. LDS: 927 accessibility-focused pictograms.

Preprocessing Steps:

• Captioning: Original labels were insufficient for diffusion training. The authors used the BLIP (Large) model to generate natural-language descriptions for every image, capturing visual attributes like object count, poses, and relationships.
• Attribute Augmentation: Leveraging ARASAAC's API, the authors systematically varied background colors, skin tones, and hair colors during training. This allows the model to learn to control these attributes at inference time.
• Instance Token: A unique synthetic token (sks) was prepended to prompts to teach the model the specific "EasyRead" style without retraining the entire base model.

B. Model Architecture and Fine-Tuning

• Base Model: Stable Diffusion v1.5.
• Adaptation Technique: LoRA (Low-Rank Adaptation) was applied to the UNet attention layers (query, key, value, and output projections).
• Parameters: Rank-16 adapters with a scaling factor of 16. All base model weights (UNet backbone, VAE, CLIP) remained frozen.
• Training: 50 epochs on an NVIDIA T4 GPU using AdamW optimizer.

C. Evaluation Framework: The EasyRead Score (ERS)

To address the lack of quantitative metrics, the authors introduced the EasyRead Score (ERS), a composite metric based on six normalized sub-metrics (weighted sum):

1. Palette Complexity ($s_{palette}$): Measures color simplicity (fewer distinct colors).
2. Edge Density ($s_{edges}$): Measures line detail (lower density preferred).
3. Saliency Concentration ($s_{saliency}$): Ensures one visually dominant object.
4. Foreground-Background Contrast ($s_{contrast}$): Measures perceptual separation.
5. Relative Stroke Thickness ($s_{stroke}$): Ensures uniform and readable outline thickness.
6. Centering Error ($s_{centering}$): Measures alignment of the main content with the image center.

Formula:
$$ERS = 0.25s_{palette} + 0.20s_{edges} + 0.15s_{saliency} + 0.15s_{contrast} + 0.15s_{stroke} + 0.10s_{centering}$$

3. Key Contributions

1. Formalization of the Task: Defined EasyRead content generation as a specific accessibility-oriented text-to-image task with explicit constraints.
2. The EasyRead Score (ERS): Introduced the first systematic, reproducible, pixel-based, and semantic framework for quantifying the quality of accessibility-oriented imagery, moving beyond purely qualitative expert reviews.
3. Open-Source Pipeline: Demonstrated that lightweight LoRA fine-tuning on a curated dataset effectively steers diffusion models to produce compliant pictograms. The code and pipeline are released as open-source.

4. Results

The authors evaluated their LoRA-fine-tuned model against the base Stable Diffusion v1.5 and two closed-source commercial systems (Global Symbols and Nano Banana Pro).

• Quantitative Performance (on 275 samples):
  • EasyRead Score (ERS): The LoRA model achieved 0.47 ± 0.06, significantly outperforming the base SD v1.5 baseline (0.40 ± 0.07). This indicates generated images are stylistically simpler and better aligned with design principles.
  • CLIP Similarity: The LoRA model achieved 31.15 ± 2.88 vs. 24.33 ± 2.75 for the baseline, demonstrating improved semantic alignment with the text prompts.
• Qualitative Observations:
  • The model reliably followed specific attribute instructions (background color, skin tone, hair color).
  • It successfully generated human figures and faces in simple scenes, whereas the base model often produced photorealistic or overly complex details.
  • While output variability exists across random seeds, the model consistently disregarded irrelevant attributes (e.g., ignoring skin color prompts when no human was present).
  • The model produced outputs comparable to, and sometimes more consistent than, closed-source commercial tools in terms of style uniformity.

5. Significance and Future Work

• Scalability: This work proves that generative AI can be adapted to produce accessible content at a fraction of the cost and time of manual design, potentially democratizing access to EasyRead materials for underserved communities.
• Evaluation Standard: The ERS provides a crucial tool for researchers to benchmark and improve accessibility-focused generative models without relying solely on expensive user studies.
• Limitations: The model struggles with complex scenes involving multiple human figures (losing facial details) and does not fully capture ethnicity beyond skin tone labels.
• Future Directions: The authors suggest integrating differentiable EasyRead components directly into the training objective and conducting structured human-centric validation studies to correlate the ERS with human perception.

In conclusion, the paper establishes that targeted, lightweight adaptation of diffusion models is a viable and effective strategy for generating high-quality, cognitively accessible visual content, bridging the gap between advanced generative AI and practical accessibility needs.