🧬 biology

Fast Protein Crystal detection and classification using Detectron2 and gaussian mixture models for continuous Bioprocessing

This paper presents an automated end-to-end pipeline integrating Detectron2-based instance segmentation and Gaussian Mixture Model clustering to achieve fast, accurate, and reproducible protein crystal detection and classification, significantly outperforming existing methods in high-density microfluidic screening while reducing observer bias.

Original authors: Raphael Kuhn, Filip Horacek, Priyanshi Kumari, Moksh Nimesh Patel, Oluwafemi Adedokun, Shahana Bano, Ketan Pancholi

Published 2026-11-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Raphael Kuhn, Filip Horacek, Priyanshi Kumari, Moksh Nimesh Patel, Oluwafemi Adedokun, Shahana Bano, Ketan Pancholi

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the quiet laboratories where new medicines are born, scientists often face a deceptively simple challenge: they must grow tiny, perfect crystals out of protein solutions. These crystals are not the sugary kind found in a kitchen, but rigid, ordered structures made of billions of protein molecules locked in place. They are essential because they act as a frozen snapshot of a molecule's shape. By shining powerful X-rays through these crystals, researchers can map the atomic architecture of proteins, revealing how they work and how drugs might stop them. This process is the backbone of modern drug discovery and structural biology. However, growing these crystals is a game of chance. The conditions required are finicky, and the resulting crystals vary wildly in size, shape, and clarity. Some are perfect gems; others are cloudy dust or needle-like shards that are useless for analysis. For decades, finding the good ones has meant a human expert staring through a microscope, sorting through thousands of images, a slow and subjective task that can miss the smallest or most subtle successes.

A team of researchers has now built a new kind of digital eye to solve this bottleneck, creating a system that can automatically find, measure, and sort these microscopic crystals with a speed and consistency no human can match. Working with a dataset of nearly two thousand microscopic images, the team trained a sophisticated computer program to recognize the unique visual signature of a protein crystal. Unlike older methods that simply drew a box around a cluster of objects or relied on basic edge-drawing tools that often got confused by bubbles or debris, this new system draws a precise outline around every single crystal, even when they are crowded together or barely visible. The researchers found that their approach could identify crystals with an accuracy that matched human experts in ninety-two percent of cases, while keeping false alarms—mistaking a speck of dust for a crystal—below five percent. This level of precision is vital because it allows scientists to trust the data without needing to double-check every image by hand.

The system does more than just count; it understands what it is seeing. After the computer isolates each crystal, it analyzes the shape and texture of the object, grouping them into categories without being told what those categories should be. In the images they studied, the system naturally sorted the crystals into four distinct groups based on their physical appearance. The largest group, making up nearly sixty percent of the findings, consisted of small crystals, reflecting the reality that most attempts produce tiny fragments rather than large, perfect gems. The other groups captured variations in size and form, from medium-sized clusters to rare, larger structures. This ability to categorize without pre-defined labels is powerful because it removes the bias of human expectation, allowing the data to reveal patterns that might otherwise go unnoticed.

To make this technology accessible to anyone in a lab, the researchers wrapped their complex algorithms into a simple, web-based tool. A scientist can upload a photo of their microscope slide, and the system instantly returns a detailed report. It highlights every crystal with a colored outline, assigns it to a specific group, and calculates its exact size in square micrometers. This transforms a tedious, hours-long process of manual inspection into a matter of seconds, freeing researchers to focus on the science rather than the sorting. The team validated their work by comparing the computer's output against the judgments of a human expert, confirming that the machine could distinguish between real crystals and artifacts like air bubbles with high reliability. By automating the detection and classification of these microscopic structures, the work offers a scalable path forward for high-throughput screening, potentially accelerating the timeline for discovering new treatments and understanding the fundamental machinery of life.

Technical Summary: Fast Protein Crystal Detection and Classification using Detectron2 and Gaussian Mixture Models

Problem Statement
Protein crystallization is a foundational step in structural biology and drug discovery, enabling the determination of atomic-level structures via X-ray diffraction. However, the manual inspection of microscopy images to identify, measure, and classify protein crystals is time-consuming, subjective, and prone to error. This bottleneck is exacerbated by high-throughput screening, which generates vast datasets where traditional manual scoring is inconsistent. Existing automated approaches face significant limitations: classical image processing methods (e.g., edge detection) fail in high-density environments due to noise and overlapping artifacts, while object detection models like YOLOv3 struggle with small objects and lack pixel-level delineation. Furthermore, supervised classification requires large, manually labeled datasets with agreed-upon categories, which are often unavailable or inconsistent across laboratories.

Methodology
The authors propose a unified, end-to-end pipeline that integrates instance segmentation with unsupervised clustering to automate protein crystal analysis. The framework consists of four primary stages:

Dataset Curation: A custom dataset of 1,860 microscopic images was curated from continuous-flow lysozyme crystallization experiments. Following quality control, 632 low-quality images were removed, yielding a final curated dataset of 1,228 images. A subset of 382 images was manually annotated with polygon masks using LabelMe for instance segmentation training. The remaining images were utilized for unsupervised classification tasks.
Instance Segmentation (Detectron2): The core detection component utilizes a fine-tuned Detectron2 Mask R-CNN model with a ResNet-101 backbone and Feature Pyramid Network (FPN). The model was initialized with COCO-pretrained weights and trained on a single class (protein crystals) to achieve pixel-accurate segmentation. Preprocessing included resizing images (short side 800px, max 1333px) and standard normalization. The training configuration used a batch size of 4 and a base learning rate of $2.5 \times 10^{-4}$ for 1,000 iterations.
Feature Extraction and Dimensionality Reduction: For each detected crystal instance, feature maps were extracted from the P3 level of the Detectron2 backbone. These high-dimensional embeddings were compressed to 64 dimensions using Principal Component Analysis (PCA) to preserve informative variance while ensuring computational efficiency.
Unsupervised Classification (GMM): The reduced feature vectors were clustered using a Gaussian Mixture Model (GMM) with four components. This approach categorizes crystals based on morphological patterns (size, shape, texture) without requiring manual labels. The pipeline outputs include segmentation masks, cluster assignments, and quantitative measurements (pixel area, calibrated area in $\mu m^2$ ).

Key Contributions
The paper outlines three main contributions:

Dataset Construction: Creation of a high-quality, annotated dataset of 382 microscopic crystal images with polygon masks, specifically designed for instance segmentation tasks.
Robust Detection Model: Development and fine-tuning of a Detectron2-based model capable of reliable multi-instance detection in challenging conditions, including high-density scenes with numerous small or overlapping crystals.
End-to-End Pipeline: Design of a complete workflow integrating segmentation, feature extraction, dimensionality reduction, and unsupervised classification, accessible via a user-friendly Gradio interface. The pipeline ensures unbiased evaluation by fitting PCA exclusively on training features.

Results

Segmentation Performance: The fine-tuned Mask R-CNN (R101-FPN) achieved an average Dice Similarity Coefficient (DSC) of 0.90 on the test set (range: 0.82–0.94). The model demonstrated robustness in distinguishing touching or partially overlapping crystals and effectively suppressed false positives from artifacts like air bubbles.
Baseline Comparison: Classical edge-based methods and YOLOv3 were evaluated as baselines. YOLOv3 failed to generalize in high-density scenarios (detecting only ~30 instances in images with >40 crystals) and produced merged bounding boxes. Classical methods frequently merged adjacent crystals or fragmented single crystals.
Classification and Clustering: The GMM identified four distinct clusters with weights [0.5926, 0.1650, 0.2252, 0.0172]. The dominant cluster corresponded to small crystals, reflecting their prevalence in the dataset. Visual inspection confirmed that these clusters correlated with distinct morphologies (size, shape, texture) without manual labeling.
Human Validation: An expert review of 50 test images (~1,200 instances) showed 92% agreement between the automated segmentation and expert annotations regarding boundary delineation. False-positive rates were below 5%, and 88% of crystals were assigned to clusters with visually consistent morphological characteristics.

Significance and Claims
The authors claim that this framework provides a scalable, reproducible solution for high-throughput crystallization screening. By combining precise instance segmentation with unsupervised classification, the system reduces observer-dependent bias and eliminates the need for extensive manual labeling. The pipeline enables accurate measurement of crystal size and morphology, supporting downstream applications in structural biology and rational drug design. The integration of a Gradio interface allows non-expert users to perform automated analysis on individual images or batches, facilitating faster protein structure determination and therapeutic discovery. The work is presented as a step toward more efficient, data-driven structural biology workflows, though the authors note that future work is required to extend the framework to temporal modeling, domain adaptation across different imaging modalities, and integration with real-time beamline safety systems.

Technical Summary: Fast Protein Crystal Detection and Classification using Detectron2 and Gaussian Mixture Models

More like this