An FPGA Implementation of Displacement Vector Search for Intra Pattern Copy in JPEG XS

This paper presents an efficient pipelined FPGA architecture with optimized memory organization for the displacement vector search module of JPEG XS's Intra Pattern Copy tool, achieving a throughput of 38.3 Mpixels/s at 277 mW to enable practical hardware deployment.

The Gist

Imagine you are trying to send a photo of a computer screen (like a spreadsheet or a video game) to a friend. Screens are tricky for computers to compress because they are full of sharp lines, text, and repeating patterns, unlike a blurry sunset photo.

To make this fast and efficient, engineers use a tool called JPEG XS. A special feature inside this tool is called Intra Pattern Copy (IPC). Think of IPC as a "Copy-Paste" superpower for images. Instead of sending the whole image again, the computer looks at the picture, finds a pattern that has already appeared (like a row of numbers), and just says, "Hey, copy that part from 50 pixels to the left."

But here is the catch: Finding the right place to copy from is hard work.

The Problem: The "Needle in a Haystack"

The computer has to search through millions of possible spots to find the perfect match. This search is called Displacement Vector (DV) Search.

• The Analogy: Imagine you are in a giant library (the image) trying to find a specific sentence to copy. You have to check every single book, every page, and every line to see if it matches. Doing this one by one is incredibly slow and uses up a lot of energy. If you try to do this in real-time (like for a video call), the computer would get overwhelmed and lag.

The Solution: A Smart Assembly Line (FPGA)

The authors of this paper built a special hardware chip (an FPGA) to solve this problem. They didn't just make the computer faster; they redesigned the entire workflow.

Here is how they did it, using simple metaphors:

1. The Assembly Line (Pipelining)

Instead of one worker checking one spot, then the next, then the next, they built a four-stage assembly line.

• How it works: Imagine a factory making sandwiches.
  • Stage 1: Someone grabs the bread.
  • Stage 2: Someone adds the meat.
  • Stage 3: Someone adds the cheese.
  • Stage 4: Someone wraps it up.
• The Magic: While the first sandwich is being wrapped, the second one is getting cheese, the third is getting meat, and the fourth is getting bread. All at the same time!
• Result: The chip can check multiple "copy spots" simultaneously, making the process incredibly fast.

2. The Smart Filing System (Memory Organization)

The biggest bottleneck was how the computer found the data.

• The Old Way (Method 0): Imagine a library where books are organized by "Room Number." To find a specific chapter, you have to run to Room 1, check a shelf, run to Room 2, check a shelf, then jump to Room 10. It's chaotic and slow because the books you need are scattered everywhere.
• The New Way (Method 1): The authors reorganized the library. Now, all the books needed for a specific "chapter" (a group of image data) are stacked neatly on one shelf in a specific order.
• The Magic: The computer doesn't have to run around the library anymore. It just grabs a whole stack of books in one smooth motion. They even added a "cheat sheet" (a Translation Lookaside Buffer) on the chip so it knows exactly where everything is without asking.

The Results: Fast, Cheap, and Efficient

Because of these two tricks (the assembly line and the smart filing system), the chip achieved amazing results:

• Speed: It can process 38.3 million pixels every second. That's fast enough for high-quality, real-time video calls without any lag.
• Power: It only uses 277 milliwatts of power. That's roughly the power of a small LED light bulb. This is crucial for battery-powered devices or devices that can't get too hot.
• Efficiency: They managed to do more work while using fewer computer parts (logic gates) than the old method.

Why Does This Matter?

This isn't just a theoretical experiment. It proves that we can build hardware that handles complex image compression in real-time without needing massive, expensive supercomputers.

The Big Picture:
This research paves the way for future devices (like ASIC chips in your phone or router) to handle high-quality, low-latency video streaming. It means clearer video calls, smoother cloud gaming, and faster transmission of screen data, all while saving energy and keeping the hardware cool.

In short: They took a slow, messy search process and turned it into a high-speed, organized assembly line, making the future of video streaming look much brighter.

Technical Summary

Here is a detailed technical summary of the paper "An FPGA Implementation of Displacement Vector Search for Intra Pattern Copy in JPEG XS."

1. Problem Statement

The Intra Pattern Copy (IPC) tool is a critical component of the JPEG XS compression standard, designed to improve coding efficiency for screen content by performing wavelet-domain intra compensation predictions. However, the core module of IPC, the Displacement Vector (DV) search, presents significant challenges for hardware deployment:

• Computational Intensity: The DV search requires traversing all possible prediction offsets to find the optimal reference that minimizes coding cost.
• Hardware Bottleneck: This iterative process creates high latency and resource demands, hindering real-time implementation in low-latency systems.
• Lack of Specialized Solutions: Existing hardware architectures for motion estimation (e.g., in H.264/HEVC) rely on pixel-domain block matching and regular memory access. They are not suitable for the group-based, frequency-domain prediction flow specific to JPEG XS IPC, where wavelet coefficients are scattered across different decomposition bands.

2. Methodology

The authors propose an optimized, pipelined FPGA architecture specifically designed for the DV search module within the JPEG XS framework. The methodology focuses on two main areas:

A. Pipelined Hardware Architecture

The system is divided into two primary engines operating in a four-stage pipeline to balance throughput and latency:

1. Residual Calculation Engine:
   • Retrieves original and reconstructed wavelet coefficients from DRAM.
   • Streams data into on-chip FIFO arrays (Q0–Q3 for original, C0–C3 for reconstructed) corresponding to four IPC Groups.
   • Computes signed residuals using parallel subtraction paths (handling sign-magnitude formats) and routes them to the next stage.
2. DV Comparison Engine:
   • Stage 0: Loads residual coefficients and generates calculation parameters (BandIdx, GrpSize, UnitWidth) based on the group index.
   • Stage 1: Computes a bitwise OR mask (GetOrMask) across the group to identify non-zero residual bits.
   • Stage 2: Calculates the GCLI (Grouped Coefficient Level Information) cost, which estimates the bitplane count of residuals. This serves as the proxy for coding cost.
   • Stage 3: Compares the current BitsTest against the historical minimum (BitsBest). If a lower cost is found, the BestDV is updated.

B. Optimized Memory Organization

To address the scattered nature of wavelet coefficients, the authors introduced a novel memory access strategy (Method 1) compared to a naive precinct-based approach (Method 0):

• Group-Aligned Storage: Instead of storing data by precinct, coefficients are organized sequentially by IPC Groups and Units.
• On-Chip TLB: A Translation Lookaside Buffer (TLB) stores the variable lengths of coefficient blocks for different groups.
• Efficiency: This allows the system to load all blocks of an IPC Unit using a single base address plus a fixed offset, eliminating complex address computations and enabling burst-based memory access.

3. Key Contributions

1. First FPGA Architecture for JPEG XS IPC: This paper presents the first hardware implementation of the DV search module specifically for the JPEG XS IPC framework.
2. Four-Stage Pipelined Design: A novel architecture that enables parallel residual computation and DV comparison across multiple IPC groups, effectively balancing throughput and latency.
3. Novel Memory Organization: The introduction of a group-aligned memory structure coupled with an on-chip TLB significantly reduces control complexity and improves memory throughput for irregular wavelet data patterns.
4. ASIC Foundation: The design provides a validated, efficient foundation for future Application-Specific Integrated Circuit (ASIC) deployment.

4. Experimental Results

The proposed architecture was implemented on a Xilinx Artix-7 (XC7A35T) FPGA running at 100 MHz.

• Performance:
  • Throughput: Achieved 38.3 Mpixels/s.
  • Latency: Maintained a low latency of 73.01 ms.
  • Power: Consumed 277 mW.
  • Power Efficiency: Improved to 138.27 Mpixels/s/W (a 6.1% increase over the baseline).
• Resource Utilization:
  • LUTs: Reduced by 7.5% compared to the baseline (12.89K vs 13.93K).
  • Flip-Flops (FFs): Reduced by 8.4% (21.79K vs 23.80K).
  • DSPs: Utilized 17 DSPs (unchanged from baseline).
  • BRAM: Increased slightly (15 vs 11) to support the optimized buffering strategy.
• Accuracy: The design achieved rate-distortion performance consistent with the IPC reference software.

5. Significance

This work is significant because it bridges the gap between the theoretical efficiency of JPEG XS's Intra Pattern Copy tool and practical hardware deployment. By solving the computational bottleneck of the DV search through specialized pipelining and memory optimization, the authors enable real-time, low-latency screen content compression on FPGA platforms. The results demonstrate that high-throughput, power-efficient hardware implementation is feasible, paving the way for ASIC adoption in professional video workflows, remote desktop applications, and KVM systems.