Floating-point consistent cross-verification methodology for reproducible and interoperable DDA solvers with fair benchmarking

This paper introduces a unified, software-assisted methodology to achieve machine-precision cross-verification and fair performance benchmarking across three major open-source DDA solvers (DDSCAT, ADDA, and IFDDA) by aligning numerical parameters and providing equivalence tables for reproducible, interoperable electromagnetic scattering simulations.

The Gist

Imagine you are trying to bake the perfect chocolate cake. You have three famous bakers: Baker DDSCAT, Baker ADDA, and Baker IFDDA. They all claim to use the exact same recipe (the laws of physics) to bake a cake that represents how light bounces off a particle.

However, when you taste the cakes, they don't taste exactly the same. One is slightly sweeter, another is a bit fluffier. Why?

• Baker DDSCAT measures flour in "cups," Baker ADDA in "grams," and Baker IFDDA in "spoons."
• Baker DDSCAT stirs clockwise, while Baker ADDA stirs counter-clockwise.
• Baker IFDDA uses a different type of oven.

For years, scientists couldn't tell if the cakes tasted different because the bakers were actually bad at their jobs, or just because they were using different measuring tools and techniques.

This paper is the "Universal Measuring Cup" project.

Here is what the researchers did, explained simply:

1. The Problem: The "Apples vs. Oranges" Mess

The three bakers (software programs) are the world's best at simulating how light interacts with tiny particles (like dust, viruses, or snowflakes). But because they were built by different teams over decades, they speak different "languages" and use different default settings.

If you asked them to simulate the same particle, they would give you slightly different answers. Scientists couldn't tell if a difference was due to a bug (a mistake in the code), a hardware difference (a faster computer), or just a translation error (one baker using a different unit of measurement).

2. The Solution: The "Translation Guide"

The authors created a translator tool (a Python software package called dda-bench). Think of this as a universal translator that forces all three bakers to speak the exact same language.

• Harmonizing the Recipe: They forced all three bakers to use the exact same amount of flour, the exact same stirring speed, and the exact same oven temperature.
• The Result: Once they did this, the cakes tasted identical. The bakers agreed on the result down to the last crumb (machine precision). This proved that the bakers weren't "bad"; they just needed to be on the same page.

3. The Race: Who is the Fastest Baker?

Once the recipes were identical, the researchers could finally have a fair race to see who was the fastest.

• The CPU Race (The Classic Oven): They tested the bakers on standard computer processors (CPUs).

  • Winner: Baker ADDA was the fastest. Why? Because ADDA figured out a clever trick to chop the big 3D mixing task into smaller 1D slices, making it much more efficient.
  • Runner-up: Baker IFDDA was good, but Baker DDSCAT was a bit slower because it didn't use the most modern mixing tools (FFT libraries).

• The GPU Race (The Super-Oven): They also tested the bakers on Graphics Processing Units (GPUs), which are like super-fast, specialized ovens used for video games and AI.

  • The Surprise: Baker IFDDA was the clear winner here. Why? Because IFDDA moved the entire baking process onto the GPU.
  • The Problem with ADDA: Baker ADDA was slow on the GPU because it kept running back and forth between the "kitchen" (CPU) and the "oven" (GPU) to check the batter. This constant running back and forth (data transfer) wasted time.
  • The Lesson: If you want to use a super-oven, you have to put the whole recipe inside it, not just part of it.

4. The "Precision" Trick

The researchers also discovered a cool trick: Single vs. Double Precision.

• Double Precision is like measuring ingredients with a microscope. It's incredibly accurate but slow.
• Single Precision is like using a standard kitchen scale. It's slightly less precise but twice as fast.

They found that for many baking tasks, the "kitchen scale" (Single Precision) was fast enough to get a perfect cake, saving a huge amount of time and energy.

Why Does This Matter?

This paper is like a rulebook for the future of light-scattering science.

1. It stops the arguing: Scientists can now compare different software fairly without worrying about "translation errors."
2. It saves money: By knowing which "oven" (CPU or GPU) and which "baker" (software) is fastest, researchers can save millions of dollars in computing costs.
3. It builds trust: We now know that these complex simulations are reliable. If two different bakers make the same cake using the same recipe, we know the cake is real.

In short: The authors built a universal translator that forced three rival software programs to speak the same language. Once they did, they could finally have a fair race, revealing that one program is best for standard computers, another is best for super-fast graphics cards, and that sometimes, being "good enough" (less precise) is actually the fastest way to get the job done.

Technical Summary

1. Problem Statement

The Discrete Dipole Approximation (DDA) is a standard numerical method for simulating electromagnetic scattering by arbitrarily shaped objects. While several open-source DDA solvers exist (notably DDSCAT, ADDA, and IFDDA), comparing their performance and accuracy has historically been difficult due to:

• Inconsistent Conventions: Differences in linear system formulations (polarization vs. symmetrized vs. internal field forms), coordinate systems, and unit conventions (SI vs. CGS).
• Default Parameter Mismatches: Variations in default polarizability models (e.g., LDR, FCD, IGT), iterative solvers, convergence thresholds, and grid definitions.
• Hidden Numerical Effects: Small discrepancies in floating-point operation ordering or solver settings can lead to divergent convergence histories, masking genuine algorithmic or hardware performance differences.
• Lack of Reproducibility: Without a unified framework, users cannot easily verify if two codes are solving the exact same numerical problem or if performance differences are due to configuration errors rather than code efficiency.

2. Methodology

The authors introduce a unified, software-assisted methodology to enable floating-point-consistent cross-verification and fair benchmarking.

• Software Framework (dda-bench): A lightweight Python command-line tool that acts as a wrapper. It executes specific command-line instructions for each solver, collects output files, extracts physical quantities, and converts them into consistent units and definitions.
• Parameter Harmonization: The core of the methodology involves aligning all "free" and "hidden" parameters across the three codes to achieve machine-precision agreement (14–15 matching digits in double precision). This includes:
  • Linear System Alignment: Mapping different mathematical formulations (e.g., ADDA's symmetrized form vs. DDSCAT's standard form) to ensure they solve the same linear system.
  • Solver Tuning: Adjusting convergence thresholds ($\eta$) and initial field guesses to ensure identical iteration counts across codes, eliminating variability caused by floating-point round-off divergence.
  • Geometric Consistency: Ensuring equivalent particle volumes and discretization grids (voxel counts) across codes, accounting for different shape definitions (e.g., equivalent radius $a_{eff}$ vs. diameter).
• Benchmarking Strategy: Once numerical equivalence is established, the framework performs systematic performance tests on:
  • CPUs: Comparing OpenMP and MPI parallelization strategies.
  • GPUs: Comparing CUDA (IFDDA) and OpenCL (ADDA) implementations, including fully offloaded solvers vs. hybrid CPU-GPU approaches.

3. Key Contributions

• Unified Equivalence Tables: The paper provides comprehensive tables (Appendix C) mapping command-line parameters between DDSCAT, ADDA, and IFDDA. This allows users to configure identical simulations across different codes.
• Machine-Precision Verification: The authors demonstrate that when parameters are strictly aligned, independent DDA implementations can agree up to machine precision (10–15 digits), proving that previous discrepancies were largely due to configuration mismatches rather than fundamental algorithmic flaws.
• Fair Performance Baseline: By removing configuration bias, the study isolates the true impact of hardware architecture (CPU vs. GPU, memory bandwidth) and algorithmic choices (FFT strategies, solver types) on runtime and scalability.
• Regression Testing Tool: The software package supports bitwise reproducibility verification for future code releases, aiding in continuous integration (CI/CD) workflows.

4. Key Results

A. CPU Performance (OpenMP vs. MPI)

• FFT Strategy Dominance: The choice of Fast Fourier Transform (FFT) backend is the primary performance driver. MKL (Intel) and FFTW significantly outperform legacy GPFA algorithms.
• Scalability:
  • ADDA (MPI-based) shows superior scalability on large clusters (AMD EPYC), achieving speedups up to 24$\times$ on 125 cores. Its 3D-to-1D FFT decomposition strategy reduces computational workload by exploiting zero-padding and symmetry.
  • DDSCAT (OpenMP-based) exhibits poor scaling on multi-core CPUs because its FFT routines are not parallelized with OpenMP; it is primarily designed for orientation averaging.
  • IFDDA scales reasonably well but generally lags behind ADDA in multi-core scenarios.
• Precision: Single-precision execution in DDSCAT offers ~60% runtime reduction on laptops with negligible accuracy loss for well-conditioned problems.

B. GPU Performance (CUDA vs. OpenCL)

• Fully Offloaded Solvers: IFDDA (CUDA), which offloads the entire iterative solver loop to the GPU, achieves the best performance. On high-bandwidth GPUs (A100, H200), it is 4$\times$ faster than the best multi-core CPU runs.
• Hybrid Limitations: ADDA (OpenCL) in its default mode offloads only FFTs, keeping the linear algebra on the CPU. This introduces significant host-device transfer latency, making it 1.7$\times$ to 1.8$\times$ slower than CPU MPI runs for large grids.
• OCL_BLAS Mode: An experimental mode in ADDA that offloads BLAS operations to the GPU (using clBLAS) reverses this trend, showing potential for GPU parity with IFDDA, though it is currently limited to the BiCG solver.
• Memory Constraints: GPU memory is the limiting factor for large grids. IFDDA requires ~2$\times$ to 4$\times$ more VRAM than ADDA for the same grid size due to full solver residency.

C. Precision Effects

• Single Precision: On workstation GPUs with low FP64 throughput (e.g., RTX 6000 Ada), switching to single precision yields speedups of 1.8$\times$ to 3.5$\times$.
• Latency Bound: On high-end cluster GPUs (H200), the solver often becomes latency-bound rather than bandwidth-bound, reducing the benefits of single precision.

5. Significance

• Reproducibility: The paper establishes a gold standard for comparing DDA solvers, moving the field from qualitative comparisons to quantitative, bitwise-reproducible benchmarks.
• Best Practices: It provides a practical guide for users to configure consistent simulations, helping them understand how to trade off precision, speed, and memory.
• Hardware Insights: The results clarify that for DDA, memory bandwidth is the critical bottleneck for GPU performance, and MPI-based 3D-to-1D FFT strategies are superior for large-scale CPU clusters.
• Future Development: The methodology supports the development of next-generation solvers by providing a rigorous framework for regression testing and verifying that code optimizations do not alter physical results.

In conclusion, this work demonstrates that modern DDA solvers are numerically mature and internally consistent. The remaining performance differences are genuine reflections of implementation efficiency and hardware architecture, which can now be fairly evaluated using the proposed cross-verification methodology.