💬 NLP

Generating Hierarchical JSON Representations of Scientific Sentences Using LLMs

This paper demonstrates that fine-tuning a lightweight LLM with a novel structural loss function to convert scientific sentences into hierarchical JSON structures enables effective information retention, as evidenced by high semantic and lexical similarity when reconstructing the original text.

Original authors: Satya Sri Rajiteswari Nimmagadda, Ethan Young, Niladri Sengupta, Ananya Jana, Aniruddha Maiti

Published 2026-03-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Satya Sri Rajiteswari Nimmagadda, Ethan Young, Niladri Sengupta, Ananya Jana, Aniruddha Maiti

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very complex, dense scientific sentence. It's like a tangled ball of yarn containing a main idea, some conditions, exceptions, and specific details all wrapped up together.

If you try to flatten that ball of yarn into a simple list (like "Subject – Verb – Object"), you often lose the shape of the knot. You might keep the words, but you lose the logic of how they connect.

This paper asks a simple question: Can we untangle that yarn, organize it neatly into a structured box, and then re-tangle it back into the original sentence without losing any meaning?

Here is the story of how they tried to do it, explained simply:

1. The Problem: The "Flat" vs. The "Tree"

Think of a scientific sentence like a family tree.

The Old Way (Flat Schema): Imagine trying to describe a family tree by just listing names in a line: "Grandpa, Dad, Me." You know who they are, but you don't know who is related to whom, or who is the parent of whom. It's too simple.
The New Way (Hierarchical JSON): This is like drawing the actual family tree. You have a "Core" (the main claim) and "Branches" (the conditions, exceptions, and details) hanging off it. This paper wanted to turn messy sentences into these neat, tree-like digital structures (called JSON).

2. The Tool: A "Smart Scribe" (The LLM)

The researchers used a "Lightweight" AI model (a smaller, faster version of the famous ChatGPT-style models called Mistral-7B). Think of this AI as a very fast, very obedient scribe.

The Training: They didn't just tell the scribe, "Write a summary." They gave it a strict rulebook. They said: "Take this sentence, find the main point, and put the supporting details in a specific folder structure. Don't just copy the words; understand the logic."
The Secret Sauce (The Loss Function): Usually, AI just tries to guess the next word. If it guesses wrong, it gets a small "ding" (a penalty).
- In this paper, they added a special referee. Every time the AI tried to output its answer, the referee checked: "Is this actually a valid, organized box (JSON)?"
- If the AI tried to spill the box or make a messy pile, the referee gave it a big penalty. This forced the AI to learn not just what to say, but how to structure it perfectly.

3. The Test: The "Magic Decoder"

Once the AI learned to turn sentences into these neat boxes, the researchers wanted to see if the boxes were useful. They didn't just look at the boxes; they tried to turn them back into sentences.

The Process: They took the neat JSON boxes and fed them into a super-smart AI (GPT-4o) and said, "Here is the organized data. Please write the original sentence back for me."
The Goal: If the new sentence sounds almost exactly like the original, it means the "box" successfully preserved the meaning. If the new sentence is gibberish or misses the point, the box failed.

4. The Results: Did it Work?

The results were surprisingly good!

The "Box" was perfect: 100% of the time, the AI produced a valid, organized structure. It learned the rules of the box perfectly.
The "Reconstruction" was strong: When they turned the boxes back into sentences, the new sentences were 87% similar in meaning to the originals.
- Analogy: Imagine you take a complex recipe, break it down into a list of ingredients and steps (the JSON), and then give that list to a chef. The chef cooks a dish that tastes 87% like the original. That's a huge success!
The Exceptions: In a few cases, the meaning got a bit fuzzy. This happened when the original sentence was so compressed or logical that the "box" couldn't hold all the nuance without getting messy.

The Big Takeaway

This paper proves that we don't have to choose between structure and meaning.

By teaching AI to organize scientific text into a "hierarchical tree" (like a well-organized filing cabinet) rather than a flat list, we can store the information efficiently and still retrieve the full meaning later. It's like proving that if you fold a map perfectly into a small square, you can still unfold it later and find your way around the whole world.

In short: They taught a computer to organize messy scientific thoughts into neat boxes, and showed that those boxes are strong enough to hold the entire meaning of the original text.

1. Problem Statement

Scientific sentences often contain complex structures, including core claims, conditional clauses, temporal qualifiers, and logical exceptions. Traditional text representation methods, such as flat "Entity–Relation–Entity" schemas, fail to capture these clause-level dependencies and hierarchical logic, leading to significant information loss. The authors address the challenge of determining whether structured, hierarchical representations can effectively preserve the semantic meaning of scientific text and allow for accurate reconstruction of the original sentence.

2. Methodology

The study employs a three-stage pipeline involving data generation, model fine-tuning with a novel loss function, and semantic reconstruction.

A. Dataset Construction

Source: 1,500 scientific sentences were collected from seven domains (Physics, CS, Math, Economics, Biology, Chemistry, Medicine) via repositories like arXiv, bioRxiv, ChemRxiv, and PubMed.
Preprocessing: Sentences containing equations, symbols, or citations were removed.
Final Set: 1,370 sentences with complete meaning were retained.
Split: 1,096 sentences for training/validation and 274 for testing.

B. Training Data Generation (GPT-4o)

High-quality training pairs were generated using GPT-4o. The prompt instructed the model to convert raw sentences into a specific hierarchical JSON format consisting of:

core: A generic label and the central claim.
hierarchy: A nested structure defining relationships between components.
Constraints: The model was instructed to avoid word-for-word copying, compress content (fields <30% of original length), and organize logic hierarchically. 17 predefined relationship types were provided as examples.

C. Model Fine-Tuning (Mistral-7B)

Base Model: Mistral-7B (a ~7B parameter LLM).
Technique: Low-Rank Adaptation (LoRA) was applied to attention and feedforward layers (Rank 16, Scaling 16, Dropout 0.05).
Novel Structural Loss Function: To ensure syntactic correctness, the authors introduced a runtime validation mechanism integrated into the training loop (Algorithm 1):
1. Standard Cross-Entropy Loss ( $L_{CE}$ ) is calculated.
2. Generated tokens are decoded and parsed as JSON.
3. If JSON parsing fails, a structure penalty ( $L_{struct}$ ) is calculated based on the failure rate.
4. Total Loss: $L_{total} = L_{CE} + L_{struct}$ .
  This forces the model to learn not just semantic content but also strict JSON syntax.

D. Sentence Reconstruction

To evaluate information retention, the structured JSONs generated by the fine-tuned Mistral-7B were fed into GPT-4o via a prompt instructing it to reconstruct a natural language sentence based on the JSON's core claim and hierarchy.

3. Key Contributions

Hierarchical JSON Schema: Proposes a structured format that separates core claims from supporting logical components, addressing the limitations of flat schemas.
Structural Loss Function: Introduces a novel training mechanism that dynamically penalizes invalid JSON outputs during fine-tuning, ensuring high syntactic validity without post-processing.
End-to-End Validation: Demonstrates a full cycle of Sentence $\to$ Structured JSON $\to$ Reconstructed Sentence, proving that the intermediate structured format retains sufficient semantic information.
Domain Diversity: Validates the approach across seven distinct scientific fields, ensuring generalizability beyond a single domain.

4. Results

A. Structural Validity

100% of the 274 test set outputs were valid JSON. This confirms the effectiveness of the structural loss function in teaching the model to produce syntactically correct structures.

B. Semantic and Lexical Similarity

The reconstructed sentences were compared against the originals using:

Cosine Similarity (Semantic): Mean score of 0.8724 (Std Dev 0.0682). Most scores remained above 0.85, indicating strong meaning retention.
Lexical Overlap Metrics:
- BLEU: 0.1496 (Low, expected due to compression and rephrasing).
- ROUGE-1 F1: 0.5725.
- METEOR: 0.4907.

C. Case Studies

High Performance: Sentence 185 (Electronic properties) achieved a Cosine similarity of 0.944 and ROUGE of 0.960, showing near-perfect retention.
Moderate Performance: Sentence 97 (Nonlocal boxes) had high semantic similarity (0.889) but lower lexical overlap, as the reconstruction reordered clauses while preserving the scientific context.
Low Performance: Sentences 134 and 522 showed lower scores (Cosine ~0.54–0.56), attributed to compressed clauses or incomplete logical content in the JSON generation.

5. Significance and Conclusion

The paper demonstrates that lightweight LLMs (7B parameters) can be effectively fine-tuned to convert unstructured scientific text into hierarchical JSON representations without losing core meaning. The high cosine similarity scores between original and reconstructed sentences validate that the hierarchical format serves as a robust intermediate representation for scientific knowledge.

Future Directions: The authors suggest exploring tree-based loss functions for better semantic role alignment and testing the system's robustness against corrupted or malformed JSON structures.

Impact: This work provides a pathway for more efficient scientific text processing, enabling better knowledge extraction, summarization, and reasoning by preserving the logical hierarchy of scientific claims rather than treating them as flat text.