UniSER: A Foundation Model for Unified Soft Effects Removal

1. Problem Statement

Digital images captured in real-world environments are frequently degraded by "soft effects"—semi-transparent occlusions such as lens flare, haze, shadows, and reflections. These effects reduce image aesthetics, contrast, and color fidelity. While the underlying scene identity often remains partially visible, recovering it is an ill-posed problem.

Current approaches suffer from two main limitations:

• Specialist Models: Existing methods address each degradation type (e.g., dehazing, shadow removal) in isolation. While effective for specific tasks, they lack scalability and fail to generalize across diverse, unseen real-world conditions.
• Generalist Models: Recent large-scale foundation models (e.g., GPT-4o, Flux Kontext, Nano Banana) offer text-driven editing but struggle with fine-grained restoration. They often rely on complex prompts, exhibit instability, and fail to preserve the precise identity of background objects, leading to "content drift" or hallucinations.

The core challenge is to develop a unified framework that can remove diverse soft effects while preserving scene identity, offering precise user control, and generalizing to undefined or unseen degradations.

2. Methodology

The authors propose UniSER, a data-centric foundation model built on a Diffusion Transformer (DiT) architecture. The methodology consists of three key pillars:

A. Large-Scale Data Curation (3.8M Pairs)

To ensure robustness and generalization, the authors curated a massive dataset of approximately 3.8 million pixel-aligned image pairs.

• Integration: They unified existing open-source benchmarks for lens flare, shadow, haze, and reflection removal.
• Data Expansion: To address gaps in public datasets (e.g., scarcity of flare data or uniform haze), they synthesized new data:
  • Lens Flare (HALO Dataset): 70k pairs rendered in Blender using 3D scenes to ensure geometric consistency and realistic light physics, covering various flare patterns (streaks, glare, shimmer).
  • Shadows (LR-SRD): 26k pairs created by stitching shadow-free objects into backgrounds to generate natural shadow-free ground truths.
  • Haze: Used a physically motivated atmospheric rendering pipeline (based on the Radiative Transfer Equation) combined with procedural noise fields to simulate non-homogeneous fog, smoke, and varying visibility levels.

B. Unified Framework Architecture

UniSER reformulates soft effect removal as a discontinuous frame generation problem within a latent diffusion model.

• Input: The model takes a noisy target latent, the input image latent, and conditional embeddings (text prompts like "remove haze" and user controls).
• Core: A Diffusion Transformer (DiT) with full attention mechanisms iteratively predicts and removes noise, conditioning on both visual context and instructions.
• Output: A denoised latent decoded by a VAE to reconstruct the clean image.

C. Controllable Editing Mechanisms

A unique feature of UniSER is its fine-grained control, trained via specific strategies:

• Random Masking Strategy: Since training data often lacks effect masks, the model is trained with randomly synthesized binary masks (combining geometric primitives and free-form strokes). This teaches the model to handle arbitrary user-defined regions.
• Removal Strength Control: Instead of binary masks, the model accepts a continuous scalar $\alpha \in [0, 1]$. The training target is generated by linearly interpolating between the input (with effects) and the ground truth (clean):
  $$I_{target} = \alpha M_{blur} \cdot I_{gt} + (1 - \alpha M_{blur}) \cdot I_{input}$$
  This allows users to specify partial removal (e.g., reducing haze by 50%) or complete removal.
• Zero-Shot Generalization: By training with a generic prompt "remove effects" and using clean images with synthetic occlusions, the model learns a universal concept of removing arbitrary semi-transparent occlusions, enabling it to handle unseen effects like rain or stains.

3. Key Contributions

1. A Large-Scale Unified Dataset: The creation of a 3.8M-pair dataset covering four major soft effect domains, including novel, physically plausible synthetic data (HALO, LR-SRD, advanced haze synthesis) to fill critical gaps in public benchmarks.
2. UniSER Foundation Model: A versatile model that outperforms both specialist models and larger generalist foundation models (like Nano Banana) in removing soft effects while strictly preserving scene identity.
3. Fine-Grained Controllability: The introduction of pixel-level mask control and continuous strength control, allowing for intuitive, precise, and partial restoration tailored to user needs.
4. Dual Capability: The framework supports not only removal but also the addition and enhancement of realistic soft effects on clean images by inverting the training process.

4. Experimental Results

The paper evaluates UniSER on standard benchmarks and challenging "in-the-wild" datasets.

• Quantitative Performance:
  • In-the-Wild: On a custom set of 39 real-world images, UniSER achieved the highest scores in reference-free metrics (LIQE, Contrast Gain) and VLM-based assessment (QwenQA), significantly outperforming specialists and generalists.
  • Standard Benchmarks: On datasets like Flare7K, SOTS, WSRD+, and SIR2, UniSER achieved State-of-the-Art (SOTA) results in PSNR and SSIM, often surpassing specialized models and matching or exceeding much larger generalist models.
• Qualitative Performance:
  • Identity Preservation: Unlike generalist models that often alter object identities or background structures, UniSER faithfully restores the original scene.
  • Robustness: The model successfully removes complex, entangled effects (e.g., dense haze with shadows) and generalizes to unseen degradations like rain and stains in a zero-shot manner.
  • Control: Visualizations demonstrate smooth transitions when adjusting removal strength and precise localization when using masks.

5. Significance

UniSER represents a significant shift in image restoration from task-specific, isolated models to a unified, controllable foundation model.

• Practical Utility: It solves the "content drift" problem common in generative AI, making it suitable for professional photo editing and critical computer vision pipelines where preserving the original scene is paramount.
• Scalability: By leveraging a unified representation of soft effects (semi-transparent occlusions), it demonstrates that diverse restoration tasks can be learned jointly, improving generalization without sacrificing performance.
• User-Centric Design: The ability to control where and how much an effect is removed bridges the gap between automated restoration and creative editing, offering a powerful tool for both automated pipelines and human-in-the-loop workflows.