3DAS: A Score to Evaluate the Fidelity of STL and Printed 3D Models: A Pilot Study

Technical Summary: 3DAS – A Score to Evaluate the Fidelity of STL and Printed 3D Models

Problem Statement
Three-dimensional (3D) printed models and their source STL files are increasingly utilized in implantology and oral/maxillofacial surgery for diagnosis, surgical planning, and procedural rehearsal. However, the clinical reliability of these workflows is critically limited by the accuracy of the models. Dimensional errors can occur at any stage of the digital pipeline—from image acquisition and segmentation to surface reconstruction and printing. Despite the prevalence of these technologies, there is no standardized, clinically oriented method for clinicians to independently verify the fidelity of STL files or printed models before use. This gap is particularly significant when workflows are outsourced to external providers, as clinicians currently lack an objective, reproducible metric to assess model quality. Furthermore, existing literature often reports isolated trueness or precision values without integrating anatomical completeness or clinical relevance, making cross-study comparisons difficult.

Methodology
The study proposes and pilots the 3-Dimensional Accuracy Score (3DAS), a composite metric designed to evaluate the anatomical fidelity and dimensional accuracy of STL files and printed 3D models relative to their originating CBCT scan.

• Score Structure: The 3DAS is a graded instrument ranging from Level I (best) to Level IV (worst). The final grade is determined by the worst performance between two components:

  1. Anatomical Fidelity: A qualitative assessment comparing the STL file or printed model against the DICOM dataset to verify the presence of predefined anatomical landmarks (e.g., foramina, crests, condyles). The grade decreases by one level for each missing or non-identifiable feature.
  2. Dimensional Accuracy: A quantitative assessment measuring the absolute percentage deviation of three orthogonal distances (sagittal, coronal, axial) from the CBCT reference. Thresholds were established based on a 0.5 mm absolute deviation (approx. 1.25% relative error for a 40 mm reference), defining Grade I as ≤2% deviation, Grade II as 2–4%, Grade III as 4–6%, and Grade IV as >6%.

• Pilot Study Design:

  • Dataset: Three anonymized maxillary CBCT scans representing low, medium, and high image quality were selected from a historical skeletal collection.
  • Providers: Seven commercial providers worldwide (including ZAGA Centers, BTK, 3DBiomodels, etc.) were commissioned to generate STL files and, where available, printed models.
  • Evaluation: Two calibrated investigators performed the primary scoring, which was subsequently verified by three independent, blinded auditors who were unaware of the provider identities.

Key Contributions

1. Development of 3DAS: The paper introduces a practical, clinically oriented framework that allows clinicians to independently verify model fidelity using only the imaging software already available in their workflow, without requiring additional equipment.
2. Identification of Fidelity Loss Sources: The study isolates the DICOM-to-STL conversion (segmentation) as the dominant source of fidelity loss, rather than the 3D printing process itself.
3. Standardized Evaluation: It provides a reproducible method to assess both anatomical completeness and dimensional accuracy simultaneously, addressing the lack of standardized metrics in current literature.

Results

• Provider Variability: The 3DAS successfully distinguished significant differences in fidelity across providers. One provider (ZAGA Centers) achieved Grade I across all three CBCT quality levels for both STL files and printed models. Conversely, several other providers consistently scored Grade IV regardless of the input image quality.
• Impact of Input Quality: For most providers, the 3DAS grade declined as the quality of the input CBCT scan decreased. However, the segmentation methodology appeared to be a more dominant factor than image quality alone, as some providers maintained high fidelity even with lower-quality inputs, while others failed with high-quality inputs.
• STL vs. Printed Models: Differences in scores between STL files and their corresponding printed models were minimal. This confirms that errors introduced during the segmentation and file conversion stages propagate directly to the printed model, and printing inaccuracies were insufficient to alter the 3DAS grades in this study.
• Reliability: Scoring was highly concordant between investigators (no discrepancy exceeding 0.5 mm), and the blinded audit confirmed identical grading outcomes, suggesting the 3DAS criteria are sufficiently defined for reproducible application.

Significance and Claims
The paper claims that the 3DAS fills a critical gap in digital dental workflows by enabling independent quality assurance prior to treatment planning. The identification of segmentation quality as the primary target for quality assurance is presented as a key finding; it suggests that clinicians cannot assume a high-quality CBCT scan will yield a high-fidelity model if the segmentation workflow is inadequate.

The authors note that low 3DAS grades have direct clinical consequences. For example, a Grade III score (4–6% deviation) could result in length discrepancies of 2–2.5 mm for a 50 mm zygomatic implant, potentially compromising safety margins near critical anatomical structures. In workflows involving patient-specific implants (PSIs), upstream segmentation errors are physically incorporated into the implant geometry, affecting fit and surgical outcomes.

The study acknowledges limitations, including the pilot nature of the research (three scans, seven providers) and the exclusion of the mandible. The authors state that future work is required to validate the 3DAS in larger datasets, extend its application to other anatomical regions, and explore whether grading thresholds need adjustment for specific complex workflows. The paper does not claim universal applicability at this stage but positions the 3DAS as a foundational tool for improving the reliability of outsourced digital workflows.