PQC-LEO: An Evaluation Framework for Post-Quantum Cryptographic Algorithms

This paper introduces PQC-LEO, an automated benchmarking framework for evaluating the computational and networking performance of Post-Quantum Cryptography algorithms across x86 and ARM architectures, revealing that higher security levels incur greater performance penalties on ARM devices compared to x86.

The Gist

Imagine the digital world is a massive, bustling city where everyone sends secret letters to each other. Right now, these letters are locked in special boxes (encryption) that only the intended recipient can open. The keys to these boxes are based on math problems that are incredibly hard for today's computers to solve, like trying to un-mix a bowl of paint back into its original colors.

The Problem: The Quantum Super-Computer
Scientists are building a new kind of computer called a "Quantum Computer." Think of it not as a faster car, but as a magic wand. With this wand, the old math problems (the locks on our boxes) become easy to solve instantly. If a bad guy gets this wand, they can open every secret letter in the city, stealing bank details, government secrets, and private messages. This is the "Quantum Threat."

The Solution: Post-Quantum Cryptography (PQC)
To stop this, cryptographers are designing new types of locks and keys. These new locks are based on different, tougher math problems that even the magic quantum wand can't break. These new systems are called Post-Quantum Cryptography (PQC).

The Challenge: The "Heavy Luggage" Problem
Here's the catch: These new locks are much bigger and heavier than the old ones.

• Old Locks: Like a tiny, lightweight key you can carry in your pocket.
• New PQC Locks: Like a giant, heavy suitcase.

If you try to put this giant suitcase on a small, battery-powered device (like a smart thermostat, a fitness tracker, or an IoT sensor), it might weigh the device down so much that the battery dies instantly, or the device moves so slowly it becomes useless. This is a huge problem for the "Internet of Things" (IoT), where billions of small devices need to stay secure.

The Hero: PQC-LEO (The "Suitcase Tester")
This is where the paper comes in. The authors, Callum Turino and his team, built a tool called PQC-LEO.

Think of PQC-LEO as a giant, automated gym and delivery testing center.

• The Gym: It tests how strong the new locks are. It asks: "How much energy does it take to lock and unlock this suitcase? How much space does it need?"
• The Delivery Center: It tests how fast these heavy suitcases can be shipped over the internet. It asks: "If I send 1,000 of these heavy suitcases, how long does it take? Does the road get clogged?"

How They Tested It
The team set up two different "delivery trucks" to test these new locks:

1. The Big Truck (x86): A powerful computer (like a desktop PC). It has a strong engine and a big gas tank.
2. The Scooter (ARM): A small, efficient device (like a Raspberry Pi, which is a tiny computer used in many smart devices). It has a small engine and a tiny battery.

They put the new PQC locks on both the Big Truck and the Scooter and started running tests.

The Results: The Scooter Struggles More
The tests revealed some interesting things:

• The Big Truck handled the heavy new locks pretty well. It was a bit slower than before, but it got the job done.
• The Scooter, however, really struggled. When they tried to use the most secure (and therefore heaviest) locks, the scooter slowed down significantly. In fact, on the small devices, the performance dropped by about 34% when switching to the highest security levels.

Why This Matters
This is like realizing that while a heavy winter coat keeps you warm, it's too heavy for a hummingbird to fly in. The paper shows us that while the new quantum-proof locks are necessary, we can't just slap them onto every device without thinking.

The Bottom Line
The PQC-LEO framework is a vital tool. It automates the process of checking if these new, heavy locks will work on our small, everyday devices. It helps engineers figure out:

• Which locks are light enough for a smart lightbulb?
• Which locks are too heavy and need to be simplified?
• How do we balance security (keeping the door locked) with speed (keeping the door open for business)?

By using this tool, we can ensure that when the "Quantum Wand" eventually arrives, our digital city is ready with new locks that are secure and practical for everyone, from giant servers to tiny sensors.

Technical Summary

Here is a detailed technical summary of the paper "PQC-LEO: An Evaluation Framework for Post-Quantum Cryptographic Algorithms."

1. Problem Statement

The rapid advancement of quantum computing poses an existential threat to current public-key cryptography (e.g., RSA, ECC) due to Shor's algorithm, which can solve integer factorization and discrete logarithm problems in polynomial time. While Post-Quantum Cryptography (PQC) algorithms have been developed to resist these attacks and standardized by NIST (e.g., ML-KEM, ML-DSA), their widespread adoption faces significant hurdles:

• Resource Constraints: PQC algorithms often involve larger key sizes, ciphertexts, and higher computational overhead, making them difficult to deploy in resource-constrained environments like IoT.
• Lack of Automated Benchmarking: Existing tools for evaluating PQC performance often require manual setup, lack scalability across different architectures (x86 vs. ARM), and fail to provide comprehensive networking performance data (specifically TLS 1.3 handshake metrics) across physical networks.
• Architecture Disparity: There is a need to understand how PQC performance degrades on different hardware architectures, particularly the performance gap between high-performance x86 servers and low-power ARM devices.

2. Methodology: The PQC-LEO Framework

The authors introduce PQC-LEO (Post-Quantum Cryptographic Library Evaluation Operator), an automated benchmarking suite designed to evaluate PQC computational and networking performance.

• Core Architecture:

  • Dependencies: Utilizes PQC implementations from OpenSSL 3.5.0, Liboqs, and the OQS-Provider (Open Quantum Safe Project).
  • Automation: Fully automates environment setup, test execution, and result parsing via Bash and Python scripts.
  • Supported Architectures: Designed to run on Debian-based systems with x86 and ARM CPU architectures.
  • Testing Scope: Supports PQC, Hybrid-PQC (combining classical and PQC), and classical algorithms within the TLS 1.3 protocol.

• Testing Modules:

  1. Computational Performance:
     • Benchmarks CPU cycles, execution time, and memory usage (Heap, Stack, extHeap).
     • Uses speed binaries from Liboqs for CPU metrics and Valgrind Massif for memory profiling.
  2. TLS Performance:
     • Handshake Testing: Measures connection establishment efficiency (connections per second) for both initial connections and session ID reuse. Uses s_server and s_time tools coordinated via netcat for synchronization between client and server.
     • Speed Testing: Measures cryptographic operation throughput (operations per second) within the OpenSSL library.
  3. Network Topology: Supports testing over localhost, virtual networks, and physical networks (using Ethernet switches), addressing a gap in existing tools that often rely solely on localhost testing.

3. Key Contributions

• Automated Framework: PQC-LEO eliminates the manual configuration burden of PQC testing, allowing researchers to easily compare algorithms across different hardware and network conditions.
• Cross-Architecture & Network Support: It is one of the few frameworks capable of conducting TLS performance tests across physically separated x86 and ARM devices, providing realistic data for IoT and edge computing scenarios.
• Hybrid-PQC Evaluation: The framework explicitly supports testing Hybrid schemes (e.g., ECDH + ML-KEM), which are critical for transitional security strategies.
• Comprehensive Data Collection: It aggregates metrics for CPU cycles, memory allocation, handshake latency, and throughput, outputting structured CSV data for analysis.

4. Experimental Results (Proof of Concept)

The authors conducted a proof-of-concept evaluation using:

• x86: HP EliteDesk 800 G3 Mini (Intel i5-6500T).
• ARM: Raspberry Pi 4 Model B (Broadcom BCM2711).

Key Findings:

• Performance Degradation on ARM: There is a significant performance reduction when implementing higher-security PQC levels on ARM compared to x86.
  • Example: On ARM, increasing security from ML-DSA-44/ML-KEM-512 to ML-DSA-87/ML-KEM-1024 resulted in a ~34% reduction in TLS handshake throughput (dropping from 5,530 to 3,672 connections/sec). The x86 processor showed a comparatively smaller degradation.
• Lattice-Based Dominance: Lattice-based algorithms (ML-KEM, ML-DSA) consistently outperformed other categories (Code-based, Hash-based) in both CPU and TLS metrics on both architectures.
• Memory Usage: ML-KEM and ML-DSA demonstrated relatively low memory footprints compared to other PQC candidates, though Hash-based schemes (like SPHINCS+) showed high signature sizes and slower generation times.
• Hybrid Performance: Hybrid-PQC schemes generally showed lower throughput than pure PQC or classical schemes due to the added computational overhead of running two algorithms, but they remain viable for transitional security.
• Algorithm Specifics:
  • ML-KEM-512 and ML-DSA-44 were the top performers in terms of speed and connection handling.
  • BIKE and FrodoKEM showed significantly higher latency and lower throughput, particularly on ARM.
  • CROSS and UOV (multivariate) showed promising results on x86 but struggled on ARM networking tests.

5. Significance and Conclusion

• Practical Deployment Insights: The study highlights that while NIST-standardized lattice-based schemes are robust, their efficiency varies drastically by hardware. The significant performance drop on ARM for higher security levels suggests that security level selection must be carefully balanced against device capabilities in IoT deployments.
• Research Tool: PQC-LEO lowers the barrier to entry for PQC research, enabling reproducible, automated, and scalable benchmarking.
• Future Directions: The authors plan to expand support to embedded platforms (e.g., via pqm4), integrate energy consumption analysis, and simulate network congestion to further refine the understanding of PQC in real-world constraints.

In summary, PQC-LEO provides a critical infrastructure for validating the practical viability of Post-Quantum Cryptography, revealing that while lattice-based schemes are generally superior, the architectural context (x86 vs. ARM) is a decisive factor in deployment success.