SafarDB: FPGA-Accelerated Distributed Transactions via Replicated Data Types

SafarDB is a novel FPGA-accelerated distributed transaction system that co-designs a network-attached replication engine with a custom FPGA network interface to achieve significantly lower latency and higher throughput for both Conflict-Free and Well-coordinated Replicated Data Types compared to state-of-the-art RDMA-based implementations.

The Gist

Imagine you are running a global bank with branches in New York, London, and Tokyo. Every time a customer deposits or withdraws money, every branch needs to know about it immediately so the account balance is correct everywhere.

This is the problem of distributed transactions: keeping data synchronized across many computers without it getting messy or broken.

The paper introduces SafarDB, a new system that solves this problem by moving the "brain" of the bank out of the main computer server and onto a specialized chip called an FPGA (Field-Programmable Gate Array) that is directly wired to the internet.

Here is the breakdown using simple analogies:

1. The Problem: The "Middleman" Bottleneck

In traditional data centers, when one computer wants to talk to another, it has to go through a long, bureaucratic process:

• The application (the bank teller) asks the Operating System (the manager) for permission.
• The manager talks to the Network Card (the mailroom).
• The mailroom sends the message across the network.
• The receiving computer has to go through the same reverse process to read it.

This is like sending a letter where you have to walk to the post office, wait in line, hand it to a clerk, who then walks it to the sorting machine, which then drives it to the other city, where it goes through another post office. It's slow and full of traffic jams.

Current high-speed systems use RDMA (Remote Direct Memory Access), which is like a "fast lane" that skips the OS manager. But even the fast lane has a toll booth: the data still has to travel from the computer's main memory to the network card via a cable (PCIe), which adds a tiny bit of delay.

2. The Solution: SafarDB (The "Super-Connected" Chip)

SafarDB changes the architecture entirely. Instead of the network card being a separate device plugged into the computer, SafarDB puts the application logic, the database, and the network card all on the same single chip.

The Analogy:
Imagine the bank teller, the vault, and the mailroom are no longer in different rooms connected by hallways. Instead, they are all built into one giant, super-fast desk.

• When a teller needs to send a message, they don't walk to the mailroom; they just slide a note across the desk to the mail slot.
• Because everything is on the same chip, the "travel time" for data is measured in nanoseconds (billionths of a second) instead of microseconds.

3. How It Handles Different Types of Transactions

The system handles two types of bank operations differently, using a "Hybrid" approach:

A. The "Easy" Stuff (Conflict-Free)

• Scenario: Two people in different branches deposit money. Since adding money doesn't conflict with adding more money, they can happen at the same time without talking to each other.
• SafarDB's Move: It uses a "Relaxed" mode. The chip updates the local balance and instantly sends the update to other branches. Because the chip is so fast, this happens almost instantly.
• Result: The system is incredibly fast (5 to 7 times faster than current top systems).

B. The "Tricky" Stuff (Conflicting)

• Scenario: Two people try to withdraw money from the same account at the exact same time. If both succeed, the account might go negative (bad!). They need to agree on who goes first.
• SafarDB's Move: This requires a "Consensus" (a vote). Usually, this is slow because computers have to wait for replies. SafarDB puts the "Voting Machine" (the consensus protocol) directly on the chip.
• The Magic: When a leader needs to change (e.g., the main server crashes), SafarDB can pick a new leader in nanoseconds. In old systems, this "permission switch" took hundreds of microseconds because it involved complex software handshakes. SafarDB does it by simply flipping a hardware switch on the chip.

4. The "Hybrid" Mode (When the Desk Gets Too Full)

FPGAs (the chips) are fast but have limited storage space (like a small desk). What if the bank has too many accounts to fit on one desk?

• SafarDB's Solution: It uses a "Hybrid" mode. The "Hot" accounts (the ones people use every day) stay on the fast FPGA chip. The "Cold" accounts (rarely used) are stored in the main computer's memory (the warehouse next door).
• SafarDB is smart enough to know which is which and only moves the data it needs, keeping the system fast even when the database is huge.

5. Why It Matters (The Results)

The paper tested SafarDB against the best existing systems (like Hamband and Waverunner).

• Speed: It is 7 to 12 times faster at processing transactions.
• Efficiency: It uses 4.5 times less electricity.
• Resilience: If a server crashes, SafarDB recovers and picks a new leader almost instantly, whereas other systems stumble and take much longer to get back on their feet.

Summary

SafarDB is like upgrading a bank from a system where tellers, vaults, and mailrooms are in separate buildings connected by slow roads, to a system where everything is built into a single, ultra-fast, self-driving vehicle. By moving the database logic directly onto the network chip, it eliminates the traffic jams of traditional computing, making distributed databases faster, cheaper, and more reliable than ever before.

Technical Summary

Here is a detailed technical summary of the paper "SafarDB: FPGA-Accelerated Distributed Transactions via Replicated Data Types."

1. Problem Statement

Data replication is essential for high availability and fault tolerance in distributed systems, but maintaining database integrity and convergence under transactional workloads requires complex coordination.

• Limitations of Current Approaches: State-of-the-art systems rely on Remote Direct Memory Access (RDMA) to achieve microsecond-scale latencies. However, traditional RDMA implementations suffer from significant bottlenecks:
  • PCIe Overhead: Communication between the CPU and the Network Interface Card (NIC) incurs latency.
  • Host Memory Intermediary: Data must pass through host memory, creating a bottleneck between the CPU and NIC.
  • Inefficiency in Consensus: Leader election and permission switching (critical for crash recovery) are slow (hundreds of microseconds) due to software overhead and lack of hardware optimization in standard NICs.
  • Rigid Architectures: Existing FPGA-based solutions (like SmartNICs) often act as "bump-in-the-wire" accelerators that offload specific verbs but do not fully co-design the application logic, replication protocol, and network stack on a single device.

2. Methodology: SafarDB Architecture

The authors propose SafarDB, a system that co-designs a network-attached FPGA accelerator with a custom network interface to execute replicated transactions directly on the FPGA.

Core Architectural Principles

1. Network-Attached FPGA: Instead of a SmartNIC attached to a host CPU, SafarDB connects an FPGA card directly to the network. The application logic, replication protocols, and RDMA stack reside on the same FPGA fabric.
2. Co-located Execution: By placing the application and the NIC on the same chip, communication replaces slow PCIe transactions with fast AXI (Advanced eXtensible Interface) streams. This enables "near-network" execution.
3. Hybrid Consistency Model: SafarDB implements Replicated Data Types (RDTs) using a hybrid approach:
   • Conflict-Free RDTs (CRDTs): Use relaxed consistency for operations that commute (e.g., counters, sets).
   • Well-coordinated RDTs (WRDTs): Use a hybrid model where conflict-free operations use relaxed consistency, while operations violating integrity constraints (e.g., bank withdrawals) trigger State Machine Replication (SMR) for strong consistency.
4. Custom RDMA Verbs: SafarDB introduces novel, FPGA-specific RDMA verbs:
   • One-sided RDMA: Directly accesses integrated FPGA storage (HBM, BRAM).
   • RPC over RDMA: Treats RDMA writes as Remote Procedure Calls (RPCs) to invoke FPGA-resident operators directly, eliminating the need for the receiving node to poll memory.
   • Write-Through: Updates both the replication log and the application state simultaneously.

Transaction Handling

• Reducible Transactions: Aggregated locally and propagated via RDMA Write or RPC.
• Irreducible Conflict-Free Transactions: Propagated via RPC to update local state directly.
• Conflicting Transactions: Serialized using an FPGA-accelerated Mu consensus protocol. The SMR module handles leader election and log replication.

Fault Tolerance & Leader Election

• Permission Switching: In traditional RDMA, changing write permissions (during leader failure) takes hundreds of microseconds. SafarDB exposes the internal Queue Pair (QP) state of the NIC directly to the SMR kernel, reducing permission switching to nanoseconds.
• Heartbeat Monitoring: Replicas monitor liveness via RDMA-exposed counters. Upon leader failure, a new leader is elected, and permissions are switched almost instantaneously.

3. Key Contributions

1. First FPGA-Accelerated RDT System: SafarDB is the first system to implement both CRDTs and WRDTs on network-attached FPGAs, supporting both relaxed and strong consistency models.
2. Novel RDMA Primitives: Introduces custom RDMA verbs that allow direct invocation of FPGA-resident accelerators and direct access to the NIC's microarchitectural state (QP registers), bypassing host CPU overheads.
3. Hybrid Execution Model: Supports a hybrid mode where data can reside in FPGA memory (for hot data) and host memory (for cold data), scaling capacity beyond FPGA limits while maintaining a single consistency interface.
4. Ultra-Fast Consensus: Demonstrates that co-locating the consensus protocol (Mu) and the NIC allows for leader election and permission switching orders of magnitude faster than software-based or traditional RDMA approaches.

4. Experimental Results

The system was evaluated on 8 AMD Xilinx Alveo U280 FPGAs connected via 100GbE, comparing against Hamband (state-of-the-art CPU/RDMA RDT) and Waverunner (FPGA-based SmartNIC consensus).

• Performance Gains:
  • CRDTs: 7.0× lower latency and 5.3× higher throughput compared to Hamband.
  • WRDTs: 12× lower latency and 6.8× higher throughput compared to Hamband.
  • YCSB/SmallBank: 8× lower latency and 5.2× higher throughput.
  • Vs. Waverunner: 25.5× lower latency and 31.3× higher throughput on YCSB (due to SafarDB handling client requests on all nodes vs. Waverunner's leader-only model).
• Fault Tolerance:
  • SafarDB detects leader failures and elects new leaders significantly faster than CPU-based systems.
  • Permission switch latency is reduced from ~hundreds of microseconds (CPU/RDMA) to 17–24 nanoseconds (FPGA).
  • Recovery overhead is minimal; SafarDB maintains higher throughput than Hamband even during leader failures.
• Power Efficiency:
  • SafarDB consumes approximately 35W, compared to 160W for Hamband (a 4.5× reduction), primarily because FPGAs avoid OS overheads and data movement costs associated with cache hierarchies.

5. Significance

SafarDB represents a paradigm shift in distributed database design by unifying the application, operating system, and microarchitecture layers on a programmable FPGA.

• Bridging the Gap: It demonstrates that FPGAs can eliminate the distinction between application logic and network protocols, achieving performance unattainable by CPU-centric architectures.
• Scalability: The hybrid mode allows systems to scale beyond the physical memory limits of a single FPGA by utilizing host memory without sacrificing consistency guarantees.
• Future Direction: The work suggests that co-designing NICs and consistency primitives on programmable hardware is a viable path for building next-generation, low-latency, high-throughput replicated transactional systems.

In summary, SafarDB proves that moving the entire replication engine and network stack onto a network-attached FPGA drastically reduces latency, increases throughput, improves fault recovery times, and lowers power consumption compared to the current state-of-the-art RDMA-based solutions.