Logics-Parsing-Omni Technical Report

This paper introduces the Omni Parsing framework and the Logics-Parsing-Omni model, which unify document, image, and audio-visual parsing through a three-level hierarchical paradigm of holistic detection, fine-grained recognition, and multi-level interpreting to transform unstructured multimodal signals into traceable, evidence-based structured knowledge.

The Gist

🧠 The Big Idea: From "Blurry Guessing" to "Forensic Detective Work"

Imagine you have a giant, messy library. Inside, there are books with complex diagrams, photos of crowded city streets, and hours of lecture videos.

Old AI models were like a student who glances at a page and says, "I think that's a chart about sales, and the guy in the video looks happy." They get the general vibe, but they often miss the specific numbers, forget who said what, or make up details that aren't there (hallucinations).

Logics-Parsing-Omni is like a super-forensic detective. It doesn't just "look" at the data; it dissects it. It breaks everything down into tiny, verifiable facts, anchors them to specific locations (like "the red arrow at 2:05 in the video"), and then uses those facts to build a logical story.

The paper introduces a new framework called Omni Parsing to turn messy, unstructured signals (like a video or a PDF) into standardized, machine-readable knowledge that is:

1. Locatable: You can point to exactly where the information came from.
2. Enumerable: You can count the facts (e.g., "There are 3 charts and 5 speakers").
3. Traceable: You can follow the logic trail from a raw pixel to a final conclusion.

---

🏗️ The Three-Step "Ladder" of Understanding

The authors built a system that climbs a three-step ladder to understand the world. Think of it like reading a complex novel:

Step 1: The "Spotter" (Holistic Detection)

• The Job: Before reading the words, you need to know where things are.
• The Analogy: Imagine a security guard scanning a room. They don't read the signs yet; they just point and say, "There is a person standing by the door," or "There is a graph on the wall."
• What it does: It finds objects, text blocks, and events in images or videos and draws a box around them with precise coordinates. It establishes the geometric baseline.

Step 2: The "Translator" (Fine-grained Recognition)

• The Job: Now that we know where things are, let's read them.
• The Analogy: This is the librarian who walks up to the boxes the guard pointed at. They read the text on the chart, transcribe the speech, and identify the brand logo on the shirt.
• What it does: It turns those boxes into data. It does OCR (reading text), ASR (transcribing speech), and extracts specific attributes (e.g., "This is a pie chart showing 40% market share").

Step 3: The "Detective" (Multi-level Interpreting)

• The Job: Connect the dots to find the meaning.
• The Analogy: This is the detective who takes the librarian's notes and the guard's map to solve the case. They don't just say, "There is a chart." They say, "Because the chart shows a drop in sales at 2:00 PM, and the speaker sounded stressed at that exact moment, the company likely had a crisis."
• What it does: It builds a reasoning chain. It links local facts to global logic, ensuring the final conclusion is backed by evidence.

---

🛠️ How They Built It: The "Two-Stage Training"

To teach the AI this skill, they didn't just dump data on it. They used a Two-Stage Training Strategy, like training an athlete:

1. Stage 1: The "Olympic Warm-up" (Panoramic Cognitive Foundation)

   • They fed the model 16 million diverse examples (images, documents, audio).
   • Goal: Make the model a generalist. It learns to recognize everything—from a cat in a photo to a math formula in a PDF. It builds a massive "encyclopedia" of visual knowledge.

2. Stage 2: The "Specialist Drill" (Unified Parsing Alignment)

   • They switched to 5 million high-quality, strict examples.
   • Goal: Teach the model to be precise. Instead of just describing a chart, it must output the data in a specific JSON format (a structured computer code). This forces the model to stop guessing and start anchoring its answers to hard facts.

---

📊 The Proof: The "OmniParsingBench"

How do you know this detective is good? You give them a test. The authors created OmniParsingBench, a massive exam covering:

• Documents: Can it read a messy PDF with tables and formulas?
• Images: Can it spot the difference between two nearly identical photos?
• Audio: Can it tell who is speaking and what background noise is happening?
• Video: Can it understand a long lecture, tracking the speaker, the slides, and the camera movements all at once?

The Results:
The new model, Logics-Parsing-Omni, beat almost everyone else, including some very expensive, closed-source giants (like Gemini-3-Pro).

• Why? Because while other models might "guess" the answer, Logics-Parsing-Omni builds its answer on a foundation of verified facts. It's less likely to lie (hallucinate) because it has to show its work.

---

💡 The "Aha!" Moment (The Ablation Study)

The paper includes a fascinating experiment to prove their point. They tried training the model in two ways:

1. Just Descriptions: Teaching it to write sentences about images (like "A happy dog").
2. Structured Parsing: Teaching it to extract the raw data first (coordinates, numbers, text) then write the sentence.

The Result: The model that only learned descriptions got worse at reasoning. It started making up facts.
The Lesson: You cannot have deep understanding without a solid foundation of raw data. Structure creates logic. If you don't anchor your thoughts to the physical reality of the image or video, your brain (or AI) will start to wander and make things up.

🚀 Summary

Logics-Parsing-Omni is a new AI system that stops treating images and videos as "blurry pictures" and starts treating them as structured databases. By forcing the AI to break down media into precise, locatable facts before trying to understand the story, it creates a much more reliable, logical, and useful tool for tasks like research, education, and data analysis.

It's the difference between a tourist saying, "That looks like a big castle," and an architect saying, "That is a 12th-century fortress with a 40-foot drawbridge, located at coordinates X, Y." The second one is ready to build something new.

Technical Summary

Here is a detailed technical summary of the Logics-Parsing-Omni technical report.

1. Problem Statement

Multimodal Large Language Models (MLLMs) face significant challenges when applied to knowledge-intensive domains involving unstructured data (documents, images, audio, and video). Current approaches suffer from structural dilemmas:

• Fragmented Definitions: Traditional pipelines often separate low-level extraction (e.g., OCR, layout analysis) from high-level semantic understanding, leading to a loss of critical context.
• Heterogeneity: Existing models struggle to unify the parsing of diverse modalities. Document parsers often reduce complex charts to mere bounding boxes, stripping semantic value. Image models tend to hallucinate details or lack fine-grained grounding. Audio-visual systems often rely solely on linear transcripts, ignoring non-speech acoustic events, speaker identity, and camera dynamics.
• Lack of Traceability: High-level descriptions often lack "evidence anchoring," meaning they cannot be traced back to specific low-level facts (pixels, timestamps, or audio segments), making them unreliable for downstream tasks like Retrieval-Augmented Generation (RAG) or automated indexing.

2. Methodology: The Omni Parsing Framework

The authors propose the Omni Parsing framework, which establishes a unified taxonomy across documents, images, and audio-visual streams. The core philosophy is to transform unstructured signals into standardized knowledge that is Locatable, Enumerable, and Traceable.

A. Three-Level Progressive Paradigm

The framework bridges perception and cognition through three hierarchical levels:

1. L1 - Holistic Detection: Achieves precise spatio-temporal grounding of objects or events to establish a geometric baseline.
2. L2 - Fine-grained Recognition: Performs symbolization (OCR/ASR) and attribute extraction on localized objects to complete structured entity parsing.
3. L3 - Multi-level Interpreting: Constructs a reasoning chain from local semantics to global logic, ensuring high-level descriptions are anchored in low-level facts.

B. Data Construction

A massive, standardized corpus was constructed covering four domains:

• Image: Includes natural images, knowledge-aware entities (landmarks, brands), and information-dense graphics (charts, geometric figures). Data is structured into JSON with precise coordinates and logical relations.
• Document: Unifies layout-faithful extraction of text, tables, formulas, and embedded visual elements (illustrations) into searchable semantic descriptions.
• Audio: Decodes audio as a multi-dimensional channel, integrating speaker-attributed transcriptions with fine-grained acoustic event detection (e.g., background noise, music) and scene descriptions.
• Video: Extends parsing into the temporal dimension, fusing visual dynamics, audio signals, and explicit camera motion (e.g., tracking, panning) into temporally aligned structured knowledge.

C. Two-Stage Training Strategy

The model, Logics-Parsing-Omni, is trained on the Qwen3-Omni-30B-A3B backbone using a two-stage progressive strategy:

1. Stage 1 (Panoramic Cognitive Foundation): A large-scale (16M samples) full-parameter SFT focusing on data scale and coverage. This stage builds "atomic capabilities" for structural parsing and basic semantic recognition without complex reasoning constraints.
2. Stage 2 (Unified Parsing Alignment): A high-precision (5M samples) balanced instruction tuning. This stage enforces strict alignment between heterogeneous inputs and standardized JSON outputs, integrating the L1–L3 progressive paradigm to ensure deep synergy between perception and cognition.

3. Key Contributions

• Unified Framework: Proposed the Omni Parsing framework, unifying perception and cognition across documents, images, and audio-visual streams via a three-level progressive paradigm.
• Standardized Dataset & Benchmark:
  • Constructed a high-quality, multi-modal corpus (16M+ samples) tailored for omni-modal parsing.
  • Released OmniParsingBench, a comprehensive evaluation suite covering six domains (Document, Natural Image, Graphics, Audio, Natural Video, Text-Rich Video) with metrics for both Perception (fidelity) and Cognition (reasoning).
• Model Release: Open-sourced Logics-Parsing-Omni, a model capable of converting complex multimodal signals into machine-readable, traceable structured knowledge.

4. Experimental Results

The model was evaluated on OmniParsingBench and public benchmarks like OmniDocBench-v1.5.

• Overall Performance: Logics-Parsing-Omni achieved state-of-the-art (SOTA) or highly competitive results across all modalities, often surpassing proprietary models like Gemini-3-Pro in Cognition metrics.
  • Graphics: Achieved an Overall score of 87.43 (vs. 86.66 for Gemini-3-Pro) and a Cognition score of 92.19, demonstrating superior logical reasoning and structural binding.
  • Text-Rich Video: Achieved an Overall score of 66.78, significantly outperforming the Qwen3-Omni baseline (25.06) and surpassing Gemini-3-Pro (61.00).
  • Audio: Set a new SOTA with an Overall score of 53.75, showing exceptional semantic comprehension in audio recognition and information extraction.
  • Document: On OmniDocBench-v1.5, the model scored 92.54, outperforming all other general-purpose MLLMs and specialized OCR models, with only a marginal gap to the specialized pipeline tool PaddleOCR-VL.
• Ablation Studies: Experiments on graphics parsing proved that fine-grained structural parsing (Perception) is essential for high-level reasoning (Cognition). Models trained only on natural language captions (without structured anchors) showed degraded logical reasoning capabilities, whereas models trained with the full structured paradigm saw massive performance leaps (e.g., Logical Reasoning in charts increased from 68.04 to 90.87).

5. Significance

• Bridging Perception and Cognition: The work demonstrates that high-level logical reasoning is not possible without rigorous, evidence-based low-level grounding. The "evidence anchoring" mechanism prevents hallucinations by forcing the model to link semantic descriptions to specific data points.
• Standardization of Unstructured Data: By converting diverse inputs (video, audio, complex docs) into a unified, machine-readable JSON format, the framework enables robust downstream applications like RAG, intelligent tutoring, and automated knowledge indexing.
• Open Science: The release of the model, dataset, and benchmark (OmniParsingBench) provides the community with robust infrastructure to further develop fine-grained multimodal parsing capabilities, narrowing the gap between open-source models and commercial closed-source systems in complex, knowledge-intensive tasks.