Descriptron-GBIF Annotator: A browser-based platform for crowdsourced morphological annotation of biodiversity images to help accelerate morphology based biodiversity data

The Descriptron-GBIF Annotator is a zero-installation, browser-based crowdsourcing platform that leverages AI-assisted segmentation and standardized ontologies to enable the public to generate structured morphological annotations from GBIF images, thereby addressing the scarcity of annotated biodiversity data through a two-tier architecture that feeds citizen science contributions into expert AI model training.

The Gist

Imagine you have a massive, global library of photographs containing millions of animals, plants, and fungi. These photos are sitting in a digital vault called GBIF (Global Biodiversity Information Facility). Right now, we know what is in the photos (e.g., "This is a beetle") and where it was found, but we don't know the details. We don't know if its wings are spotted, if its legs are hairy, or exactly how its body parts are shaped.

The problem is that there aren't enough expert scientists (taxonomists) to look at every single photo and write down these details. It's like having a billion books on a shelf, but only a few librarians to read them.

Enter the "Descriptron-GBIF Annotator."

Think of this tool as a digital "connect-the-dots" game for nature lovers, but with a super-smart AI assistant helping you out. Here is how it works, broken down into simple concepts:

1. The "Magic Paintbrush" (AI Assistance)

Usually, drawing around a tiny insect leg on a computer screen is frustrating and takes forever. This tool uses a super-powerful AI called SAM2 (Segment Anything Model 2).

• The Analogy: Imagine you are coloring a picture in a coloring book. Normally, you have to stay perfectly inside the lines with a tiny crayon. With this tool, you just tap the picture with your finger or mouse, and the AI instantly says, "Oh, you want the antenna? Done!" It automatically highlights the whole antenna for you. You just tweak it if it gets a little messy.

2. The "Rosetta Stone" for Body Parts (Templates & Ontology)

Once the AI highlights the part, you need to tell the computer what it is.

• The Analogy: Think of the tool as having a set of transparent plastic overlays (templates) for different animals. If you are looking at a beetle, the tool puts a template over the image showing where the "head," "wing," and "leg" go. If you are looking at a mushroom, it shows where the "cap" and "stem" are.
• When you label a part, you aren't just writing "leg"; you are linking it to a universal scientific dictionary (an ontology). This ensures that a scientist in Germany and a student in Brazil both mean the exact same thing when they say "antenna."

3. The "Two-Tier" Teamwork System

The paper describes a clever two-part system that works like a training school for AI:

• Tier 1: The Public Playground (The Annotator)
  This is the free, easy-to-use website where anyone (hobbyists, students, birdwatchers) can play. You upload a photo, the AI helps you draw lines, and you label the parts. This is the "crowdsourcing" part—getting thousands of regular people to do the heavy lifting.
• Tier 2: The Pro Lab (The Portal)
  This is a high-tech, password-protected room for professional scientists. They use super-computers to take the thousands of drawings made by the public and use them to teach the AI how to get even better.
• The Feedback Loop: The public teaches the AI, the AI gets smarter, and the smarter AI helps the public annotate even faster. It's a virtuous cycle where everyone gets better at the job.

4. The "Time-Travel" Machine (Publishing)

When you finish annotating a picture, the tool doesn't just save it to your hard drive. It instantly packages it up and sends it to Zenodo, a digital library that gives your work a permanent ID card (a DOI).

• The Analogy: It's like finishing a puzzle and immediately having it framed, hung in a museum, and given a plaque with your name on it. Your contribution becomes a permanent part of scientific history that anyone can find and use.

Why Does This Matter?

Right now, we have a "biodiversity crisis." We are losing species faster than we can describe them. We have millions of photos, but they are "dumb" data—they don't tell us about the creature's shape or features.

This tool turns those "dumb" photos into "smart" data. By turning millions of images into structured, searchable information, we can:

• Discover new species faster.
• Track how animals are changing over time.
• Let computers learn to identify species automatically.

In short: This paper introduces a tool that lets anyone become a citizen scientist, using AI to turn simple photos into a massive, global database of biological knowledge, helping us solve the mystery of life on Earth before it's too late.

Technical Summary

1. Problem Statement

The paper addresses a critical bottleneck in biodiversity science: the "taxonomic impediment."

• The Gap: Conservative estimates suggest 80% of Earth's species remain undescribed, particularly among hyperdiverse "dark taxa" (e.g., arthropods). There is a severe shortage of professional taxonomists to describe them.
• Data Paradox: While the Global Biodiversity Information Facility (GBIF) hosts over 2.7 billion occurrence records, the vast majority lack structured morphological data. They consist mainly of locality, date, and taxonomic labels, with no detailed anatomical descriptions.
• Limitations of Existing Tools: Existing citizen science platforms (e.g., iNaturalist, Notes from Nature) excel at occurrence data and label transcription but lack the capability for structured morphological annotation (identifying specific body parts and their traits).
• Technical Barrier: High-quality morphological annotation traditionally requires desktop software, domain expertise, and manual labor, making it inaccessible to non-specialists.

2. Methodology

The authors present a two-tier architecture designed to bridge the gap between public engagement and professional taxonomy, centered on a browser-based tool.

A. The Descriptron-GBIF Annotator (Tier 1: Public)

This is a zero-installation, client-side web application designed for crowdsourced annotation.

• Architecture: Implemented as a single, self-contained HTML file (~270 KB). It requires no server-side processing for the user, runs entirely in the browser, and can be hosted on any static server or opened locally.
• Data Integration:
  • GBIF: Queries the GBIF Occurrence API to retrieve specimen images, metadata, and taxonomic classifications.
  • Zenodo BioSysLit: Integrates with the Biodiversity Literature Repository to allow annotation of figures from taxonomic publications (e.g., ZooKeys, Zootaxa).
• AI-Assisted Segmentation (SAM2):
  • Utilizes Segment Anything Model 2 (SAM2) for zero-shot segmentation.
  • Hybrid Deployment: A FastAPI backend (Hetzner cloud) runs the SAM2 encoder to generate image embeddings. The browser runs the ONNX mask decoder via onnx-runtime-web to perform real-time inference (<100ms) based on user prompts (points, boxes).
• Morphological Templates & Ontology:
  • Templates: 25 taxonomic groups (covering insects, vertebrates, plants, fungi, etc.) with 124 standardized views (e.g., habitus lateral/dorsal/ventral, head frontal).
  • Ontology Linking: Every anatomical region is linked to formal ontologies: UBERON (general anatomy), HAO (Hymenoptera Anatomy), and Plant Ontology (PO).
  • Attributes: 335 unique ontology CURIEs and 745 possible instances, with controlled vocabularies for traits like texture, color, and setae density.
• Annotation Tools: Includes point prompts, bounding boxes, configurable paintbrushes, erasers, keypoint placement, line demarcation (for sutures), and a scale bar calibration tool.
• Export Formats: Supports Darwin Core (DwC) JSON, COCO JSON (for computer vision), Traits CSV, and a novel JSON-LD Knowledge Graph.
• Publishing: A built-in pipeline allows users to deposit annotated datasets directly to Zenodo with DOIs, ensuring FAIR (Findable, Accessible, Interoperable, Reusable) compliance.

B. The Descriptron Portal (Tier 2: Professional)

A GPU-accelerated workbench for professional taxonomists.

• Infrastructure: Runs on Hetzner CPU VPS with on-demand GPU pods (RunPod) via Docker and Apache Guacamole for browser-based VNC access.
• Advanced Tools:
  • Fine-tuning: LoRA adapters for SAM2-PAL (Palindrome-based mask propagation) and DINOSAR v2 (contrastive species-level embeddings).
  • Morphometrics: Generalized Procrustes Analysis (GPA) for geometric morphometrics.
  • Species Delimitation: Multi-modal Bayesian approaches fusing image embeddings, morphological traits, and DNA barcoding.
  • Automation: GPT-4 API integration for generating formal species descriptions.

C. The Feedback Loop

The system creates a bidirectional data flow:

1. Public to Pro: Crowdsourced annotations (Tier 1) are exported as COCO JSON and ingested by Tier 2 to train and fine-tune AI models.
2. Pro to Public: Expert-validated annotations from Tier 2 are exported and loaded into Tier 1 for review, knowledge graph conversion, and archival, improving the public tool's baseline.

3. Key Contributions

• First Browser-Based Morphological Annotator: A zero-install tool that brings structured morphological data collection to the general public using AI assistance.
• Ontology-Linked Data: Moves beyond simple image tagging to structured, ontology-based data (UBERON/HAO) that is machine-readable and interoperable.
• Novel Knowledge Graph Export: Generates JSON-LD files linking specimens, taxa, images, anatomical regions, and traits, creating a scalable morphological knowledge base.
• Two-Tier Scalable Architecture: A novel framework where citizen science generates training data for professional AI models, which in turn refine the tools used by the public.
• BioSysLit Integration: Enables the extraction of structured data from existing taxonomic literature figures, not just museum specimens.

4. Results

• Functionality: The system successfully retrieves images from GBIF and Zenodo, auto-detects taxonomic groups to load appropriate templates, and performs real-time segmentation using SAM2 in the browser.
• Coverage: The template library covers 25 major taxonomic groups with 124 views, ensuring standardized data collection across diverse life forms.
• Interoperability: The tool successfully exports data in Darwin Core, COCO, and JSON-LD formats, demonstrating compatibility with existing biodiversity informatics infrastructure and computer vision frameworks (Detectron2, OpenCV).
• Usability: The mobile-responsive design and simplified interface allow non-experts (students, hobbyists) to contribute valid morphological data.

5. Significance

• Solving the Taxonomic Crisis: By leveraging the "long tail" of citizen scientists, the tool scales morphological data collection beyond the capacity of professional taxonomists alone.
• Transforming Specimen Images: It converts millions of static GBIF images into rich, machine-readable morphological resources, addressing the scarcity of structured data.
• FAIR Data Principles: The direct integration with Zenodo and use of standard ontologies ensures that morphological data is Findable, Accessible, Interoperable, and Reusable.
• Future of Biodiversity Informatics: The paper proposes a paradigm shift where AI models are continuously improved by human-in-the-loop crowdsourcing, creating a sustainable ecosystem for describing Earth's biodiversity.

Availability:

• Tool: https://descriptrongbifannotator.org
• Portal: https://descriptronportal.org
• Code: GitHub (Open Source) and Zenodo.