kikuchipy: an open-source toolbox for analysis of EBSD patterns

This paper introduces kikuchipy, an open-source Python toolbox for analyzing electron backscatter diffraction (EBSD) patterns that supports Hough and dictionary indexing, orientation refinement, and validation, demonstrated through applications in stainless steel and aluminum alloy microstructure characterization.

The Gist

Imagine you are a detective trying to solve a mystery inside a tiny, invisible world. The "suspects" are the microscopic crystals that make up metals and alloys. To catch them, you use a special camera inside a powerful microscope called an Electron Backscatter Diffraction (EBSD) machine. When you shine an electron beam on a metal sample, it bounces back and creates a complex, glowing pattern of lines and bands on a screen. These patterns are like unique fingerprints for every type of crystal.

The problem is that reading these fingerprints is incredibly hard. It's like trying to solve a jigsaw puzzle where the pieces are blurry, the lighting is bad, and some pieces look almost identical to others. Usually, scientists have to use expensive, "black box" software to solve these puzzles. You put the data in, and the machine spits out an answer, but you can't see how it solved it, and if the answer is wrong, you have no idea why.

Enter "kikuchipy."

Think of kikuchipy as a new, open-source "Swiss Army Knife" for these detectives. It's a free toolbox written in the Python programming language that lets scientists take apart the puzzle-solving process step-by-step. Instead of a magic black box, it gives you a clear, transparent workbench where you can tweak, test, and improve every move you make.

Here is how the paper explains what this toolbox can do, using simple analogies:

1. Cleaning Up the Messy Photo

Before you can solve the puzzle, you often have to clean the photo. The raw patterns from the microscope can be noisy or have a hazy background (like a photo taken through dirty glass).

• The Analogy: Imagine taking a photo of a starry night, but there's a thick fog and a streetlight glare washing out the stars.
• What kikuchipy does: It has tools to subtract that "fog" (background correction) and sharpen the image. It can even take a blurry photo and blend it with its neighbors to make the stars (the crystal lines) pop out clearly.

2. Calibrating the Camera

To know exactly where a crystal is pointing, you need to know exactly where the camera is standing relative to the sample.

• The Analogy: If you are trying to map a city, you need to know exactly where your compass is pointing and how far you are from the buildings. If your compass is off by a few degrees, your map will be wrong.
• What kikuchipy does: It helps you "calibrate" the camera's position (called the "projection center") so the map matches reality. It can even adjust this position for every single point in the map, like a GPS that updates its location as you drive.

3. Solving the Puzzle (Indexing)

Once the image is clean and the camera is calibrated, you have to match the pattern to a library of known crystals.

• The Analogy: Imagine you have a library of 10,000 different fingerprints. You have a blurry print from the crime scene, and you need to find the match.
• Two ways to do it:
  • Hough Indexing: This is like quickly scanning the library for the general shape of the lines. It's fast but might miss subtle details.
  • Dictionary Indexing: This is like comparing the entire blurry print against every single fingerprint in the library, pixel by pixel, looking for the perfect match. It's slower but much more accurate, especially for tricky cases.
• The Refinement: If the match is close but not perfect, kikuchipy can "nudge" the answer slightly to find the exact fit, like adjusting a radio dial until the static disappears and the music is clear.

4. The "Truth Check"

The most powerful part of kikuchipy is that it lets you double-check your work visually.

• The Analogy: Instead of just trusting the computer's answer, you can take the computer's "best guess" and project a perfect, simulated version of what that crystal should look like. Then, you put the real photo and the simulation side-by-side.
• What it shows: If the lines and shadows in the simulation line up perfectly with the real photo, you know you solved it right. If they don't match, you know you made a mistake and can go back and fix it.

Real-World Cases from the Paper

The authors tested this toolbox on three difficult metal mysteries:

1. The "Super" Steel: They looked at a super-strong steel that had developed unwanted, brittle crystals inside it. Using kikuchipy, they could map exactly where these bad crystals formed and how they were oriented relative to the good ones. It was like seeing the blueprint of a building's weak spots.
2. The Aluminum vs. Silicon Mix: In a common metal alloy, Aluminum and Silicon look almost identical under the microscope because their crystal structures are so similar. It's like trying to tell apart two identical twins wearing the same clothes. Most software gets confused. But because kikuchipy looks at the brightness of the lines (not just their shape), it could successfully tell the twins apart and map where the Silicon was hiding.
3. The Noisy Alloy: They looked at a metal that had been crushed and rolled so hard that the crystal patterns were very fuzzy and noisy. It was like trying to read a book in a hurricane. By using the toolbox to clean the noise and compare the patterns carefully, they could still identify the tiny particles inside, even when the signal was very weak.

The Big Picture

The paper concludes that kikuchipy isn't just about solving puzzles faster; it's about solving them better and understanding how you solved them. It is built for the scientific community to share, improve, and adapt. It turns EBSD analysis from a "trust the machine" process into a transparent, flexible, and collaborative investigation, allowing anyone to peek behind the curtain and see the crystal world clearly.

Technical Summary

Technical Summary: kikuchipy: an open-source toolbox for analysis of EBSD patterns

Problem Statement
Electron Backscatter Diffraction (EBSD) is a standard technique for characterizing crystallographic features in scanning electron microscopy (SEM). However, the indexing of EBSD patterns (EBSPs) is susceptible to various sources of error, including mis-indexing of crystallographic phases, scattering of orientation points within grains, and difficulties in differentiating phases with similar crystal structures. While commercial solutions exist, they often prioritize automation and user-friendliness at the expense of flexibility and transparency. This "black box" approach limits the ability of researchers to troubleshoot problematic cases, optimize indexing steps, or understand the underlying algorithms. Although open-source alternatives exist (e.g., EMsoft, EMSphinx, AstroEBSD, PyEBSDIndex), there remains a need for a widely accessible, flexible Python-based toolbox that integrates seamlessly with the scientific Python ecosystem to facilitate iterative workflow development and result validation.

Methodology
The paper introduces kikuchipy, an open-source Python toolbox designed for the analysis of EBSPs. The software extends the HyperSpy library for multi-dimensional dataset analysis and relies on orix for handling rotations and crystal symmetry, and diffsims for diffraction simulations.

The core methodology emphasizes a flexible, iterative offline indexing workflow (illustrated in Fig. 2 of the paper), which includes:

1. Data Loading and Inspection: Reading files from major vendors (Bruker, EDAX, NORDIF, Oxford) and generating "pre-indexing" maps (e.g., mean pattern intensity, virtual backscatter electron images, image quality maps) to guide phase selection and validate results without user bias.
2. Pattern Processing: Applying static and dynamic background corrections, noise reduction via neighbor averaging, and binning to improve signal-to-noise ratio (SNR) or speed.
3. Geometry Calibration: Determining the projection center (PC) and detector-sample geometry. This involves Hough indexing for initial estimates and refinement via pattern matching. The software supports both fixed and dynamic PC models.
4. Indexing:
   • Hough Indexing: Implemented via PyEBSDIndex for rapid orientation estimation based on Kikuchi band detection.
   • Dictionary Indexing: Adapted from EMsoft, this method compares experimental patterns against a dictionary of dynamically simulated master patterns. It utilizes normalized cross-correlation (NCC) as the similarity metric.
5. Refinement: Orientation and/or projection center refinement is performed to improve accuracy, optimizing the NCC score using algorithms from SciPy and NLopt.
6. Validation: Results are validated by visually comparing experimental patterns with geometrical, kinematical, and dynamical simulations, as well as by analyzing Inverse Pole Figure (IPF) maps.

The software is designed to handle datasets larger than available memory using Dask and utilizes Numba for just-in-time compilation of performance-critical sections.

Key Contributions and Results
The paper demonstrates the capabilities of kikuchipy through three distinct application examples:

1. Orientation Relationships in Super Duplex Stainless Steel (SDSS):

   • The workflow was applied to an SDSS containing undesirable secondary phases (sigma, chi, R, $\pi$, and chromium nitrides).
   • Pre-indexing maps successfully identified microstructural features, including lamellar structures and high-Z contrast particles.
   • Dictionary indexing distinguished between multiple phases, including those typically requiring Transmission Electron Microscopy (TEM) due to size (down to ~0.52 $\mu$m).
   • The study visualized orientation relationships (ORs) between phases (e.g., $\alpha$/$\gamma$, $\alpha$/$\sigma$, $\alpha$/$\chi$) using misorientation distributions in axis-angle space. It confirmed the Kurdjumov-Sachs relationship for $\alpha$/$\gamma$ and identified a "Cube-on-cube" OR for $\alpha$/$\chi$, as well as a previously unreported CSL3 relationship between chi and ferrite.

2. Phase Differentiation in Al–Si Alloys:

   • The challenge of differentiating between Aluminum (FCC) and Silicon (Diamond cubic), which have similar Kikuchi band positions, was addressed.
   • The paper demonstrates that dictionary indexing using dynamical simulations, which leverage relative intensity distributions rather than just band positions, can successfully differentiate these phases based on EBSPs alone.
   • Validation involved comparing experimental patterns to simulated patterns, showing that intensity mismatches in Al simulations (vs. Si) clearly indicated the correct phase.
   • The analysis revealed texture in silicon fibers and the absence of a strong orientation relationship between Al and Si in the modified alloy.

3. Phase Differentiation in Noisy Al–Mn Alloys:

   • The software was used to index micron-sized constituent particles ($Al_6Mn$ and $\alpha$-AlMnSi) in a heavily cold-rolled and annealed Al–Mn alloy where EBSPs suffered from low SNR due to stored energy and camera noise.
   • By combining background correction, neighbor averaging, and dictionary indexing, the software successfully identified and distinguished between the two particle phases.
   • The results were validated against an independently generated "ground truth" map derived from Backscattered Electron (BSE) imaging.
   • The analysis revealed a preferred orientation for $Al_6Mn$ particles and a large, systematic lattice rotation within a specific $\alpha$-AlMnSi particle, attributed to heavy deformation.

Significance and Claims
The paper positions kikuchipy not as a tool for maximizing indexing speed, but as a resource for the electron microscopy community to improve flexibility, transparency, and result validation.

• Flexibility and Iteration: The software allows users to build custom workflows where indexing steps can be optimized and sources of error identified, addressing the rigidity of commercial solutions.
• Validation: It emphasizes the importance of visual validation (comparing experimental and simulated patterns) and independent verification (e.g., using pre-indexing maps or BSE images) to ensure indexing accuracy.
• Open Science: The code is hosted on GitHub, and the raw data and workflow notebooks for the examples are made publicly available on Zenodo, adhering to FAIR principles.
• Community Resource: The authors state that the software is developed to allow anyone to improve the code or integrate it into their own analysis workflows, fostering collaboration and advancement in EBSD analysis.

The paper concludes that while current limitations exist (such as the need for manual setup per phase for dictionary indexing and the lack of a native GUI), the integration of Python's scientific ecosystem makes kikuchipy a powerful tool for developing robust and flexible EBSD analysis workflows.