Evalet: Evaluating Large Language Models through Functional Fragmentation

1. Problem Statement

Practitioners increasingly rely on LLM-as-a-Judge approaches to evaluate generative AI outputs. However, current methods suffer from significant limitations:

• Holistic Opacity: Existing evaluators produce a single numeric score (e.g., 3/5) and a brief justification for an entire output. This obscures which specific elements influenced the assessment.
• Lack of Granularity: When multiple outputs receive identical scores, practitioners cannot easily determine if they share the same underlying issues or different ones.
• Validation Burden: To trust or correct an LLM evaluator, humans must manually scan entire outputs to map justifications to specific text fragments. This manual process negates the efficiency benefits of automation, especially at scale.
• Inability to Detect Systemic Patterns: The lack of fine-grained data prevents practitioners from identifying recurring behavioral patterns (e.g., a model consistently using harmful metaphors) across large datasets.

2. Methodology: Functional Fragmentation

The authors propose Functional Fragmentation, a novel evaluation paradigm that dissects LLM outputs into key text fragments and interprets the rhetorical function each fragment serves relative to specific evaluation criteria.

Core Concept

Instead of scoring the whole text, the system:

1. Extracts relevant text fragments.
2. Interprets the function of each fragment (e.g., "personification," "war imagery," "logical transition").
3. Rates whether that specific function aligns with the evaluation criterion (Positive/Negative).
4. Clusters similar functions across different outputs to reveal patterns.

The Evalet System

Evalet is an interactive system that instantiates this approach. Its technical pipeline includes:

• LLM-Based Extraction & Evaluation:
  • Uses a prompt chain (specifically optimized for Claude 3.7 Sonnet) to review an output, extract criterion-relevant fragments, analyze their function, and assign a rating.
  • The prompt instructs the LLM to "review aloud," extract fragments, analyze them, abstract them into function labels, and rate them based on user-defined criteria and few-shot examples (positive/negative/excluded).
• Multi-Level Clustering:
  • Embedding: Function descriptions are converted to embeddings.
  • Base Clustering: Uses HDBSCAN to group similar functions into "Base Clusters" (e.g., "Creative Wordplay").
  • Super Clustering: Uses KMeans to group base clusters into "Super Clusters" (e.g., "Creative Marketing Strategies").
  • Deduplication & Reassignment: An LLM is used to merge redundant super clusters and reassign base clusters to ensure semantic consistency.
• Visualization Interface:
  • Map Visualization: Projects function embeddings into a 2D space (using UMAP). Points represent functions; dots indicate positive ratings, crosses indicate negative. Clusters are visualized as contours.
  • Information Panel: Allows users to inspect specific outputs, view fragment-level justifications, and manage "Selected Entries" (examples) to refine criteria.
  • Interactive Feedback Loop: Users can re-rate specific functions or add them to "positive," "negative," or "excluded" example sets. The system re-runs evaluations using these few-shot examples to steer the LLM evaluator.

3. Key Contributions

1. Functional Fragmentation Approach: A shift from holistic scoring to a qualitative, fine-grained analysis where outputs are decomposed into functional roles (rhetorical purposes) rather than just lexical content.
2. Evalet System: An interactive tool that supports Inspect (viewing fragment functions), Rate (re-rating specific functions or adding examples), and Compare (visualizing function clusters across outputs).
3. Inductive Coding Support: The system mimics the human process of inductive coding, surfacing emergent codes (functions) that help practitioners concretize abstract evaluation criteria.
4. Calibrated Trust: A mechanism for practitioners to verify the "why" behind an LLM's judgment, allowing them to calibrate trust rather than blindly accepting or rejecting scores.

4. Results

Technical Evaluation

• Fragment Extraction: Compared against a standard "Rating" baseline (LLM gives a score + justification), Evalet's functional fragmentation approach achieved significantly higher Recall (90.2% vs. 84.3%) and F1-score (0.726 vs. 0.711) in identifying relevant text fragments.
• Overall Assessment: In identifying higher-quality outputs from pairs (RewardBench dataset), Evalet outperformed the baseline (80.1% accuracy vs. 75.5%), demonstrating that fragment-level analysis captures nuance better than holistic scores.

User Study (N=10)

A within-subjects study compared Evalet (Fragmented) against a baseline providing only Holistic scores/justifications.

• Validation: Participants identified 48% more evaluation misalignments (inconsistencies or errors in the LLM's judgment) in the Fragmented condition.
• Trust Calibration: Participants developed "informed trust." They could verify why an evaluation was made, leading to more selective reliance on the evaluator. In the Holistic condition, users often disregarded evaluations entirely due to lack of verifiability.
• Actionability: Participants rated issues identified in the Fragmented condition as significantly more actionable (p=0.04). They could pinpoint specific text to fix rather than vague "writing quality" issues.
• Workload: Perceived workload (NASA-TLX) was statistically similar between conditions, though the distribution of effort differed (Fragmented focused on comparison/exploration; Holistic focused on manual reading).
• Limitations in Correction: While Fragmented helped identify issues, correcting them was sometimes harder because the system extracted full sentences/paragraphs as examples, whereas users sometimes wanted to correct only a specific phrase.

5. Significance and Future Directions

• Shift in Evaluation Paradigm: The paper argues for moving LLM evaluation from opaque, quantitative metrics to qualitative, interpretable, and fine-grained analysis.
• Handling Scale: As LLM outputs become longer and more complex (e.g., agentic workflows), holistic scores become increasingly useless. Functional fragmentation allows practitioners to break down and interpret these complex outputs.
• Human-in-the-Loop: Evalet demonstrates how interactive systems can bridge the gap between automated evaluation and human sensemaking, turning "black box" scores into transparent, verifiable insights.
• Future Work: The authors suggest integrating priority weighting for fragments, supporting larger datasets via recursive clustering, and refining the granularity of user control over fragment extraction.

In summary, Evalet transforms LLM evaluation from a "black box" scoring mechanism into a transparent, interactive sensemaking tool, enabling practitioners to understand, verify, and correct model behaviors at a granular level.