💬 NLP

Mapping the Web of Science, a large-scale graph and text-based dataset with LLM embeddings

This paper demonstrates the practicality of combining LLM embedding models with graph-based analysis to map the semantic landscape of the Web of Science dataset, which contains approximately 56 million scientific publications.

Original authors: Tim Kunt, Annika Buchholz, Imene Khebouri, Thorsten Koch, Ida Litzel, Thi Huong Vu

Published 2026-02-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tim Kunt, Annika Buchholz, Imene Khebouri, Thorsten Koch, Ida Litzel, Thi Huong Vu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the entire world of scientific research as a massive, chaotic library containing over 56 million books (scientific papers). For a long time, librarians have tried to organize this library by looking at the "footnotes" and "citations"—essentially, who is quoting whom. If Paper A quotes Paper B, they are neighbors. This is like organizing a library by seeing which books are often checked out together. It's a useful map, but it only shows the connections between books, not what the books are actually about.

This paper introduces a new way to map this library using a "super-smart reader" (an AI called a Large Language Model, or LLM). Instead of just looking at footnotes, this reader actually reads the abstracts (the summaries) of the papers to understand their meaning.

Here is the breakdown of their approach using simple analogies:

1. The Two Ways to Map the World

The authors argue that we need to combine two different ways of understanding science:

The Graph (The Web of Connections): This is the old way. It looks at the structure: "Who cited whom?" It's like drawing a map based on which cities have roads connecting them.
The Text (The Meaning): This is the new way. It uses AI to read the content and understand the ideas. It's like looking at a map based on the language spoken in each city.

The paper suggests that while the "road map" (citations) is good, the "language map" (AI reading) can reveal the true shape of the landscape, showing us how ideas naturally group together based on what they actually say.

2. Turning Words into Coordinates

To make this work, the researchers used a tool that turns every sentence into a set of numbers (a vector).

The Analogy: Imagine every scientific paper is a drop of water. The AI drops each one into a giant, invisible 3D (or even 1,000-dimensional) globe.
The Result: Papers about the same topic (like "Quantum Physics") naturally float close together, forming a cluster. Papers about "Ancient History" float far away.
The Shape: The authors describe this as a "point cloud" that looks like continents on a globe. If you were to draw a map of these "continents," you would see distinct landmasses for Natural Sciences, Social Sciences, and Humanities.

3. Testing the Map

The team didn't just guess; they tested if this new "meaning map" matched the old "citation map."

The Correlation: They found a positive link. Generally, if two papers are cited together often (close on the road map), they also talk about similar things (close on the language map).
The Surprise: However, the link wasn't perfect (about 33% to 45% correlation). This is actually good news! It means the two maps see different things.
- Interdisciplinary Papers: Some papers are about two very different topics. On the "road map," they might be far apart because they don't share many citations. But on the "language map," the AI sees the connection because it reads the words.
- The Hybrid Approach: The authors propose that the best map is a combination of both. By using the "road" data and the "language" data together, you get a more accurate picture of the scientific world.

4. The "Soft" Boundaries

One of the most interesting findings is that you can't draw hard lines between scientific fields.

The Analogy: Imagine the continents on our globe don't have sharp borders like a political map. Instead, they have fuzzy edges where the land slowly turns into water.
The Reality: A paper might be 60% "Biology" and 40% "Chemistry." The authors found that because scientists often mix topics, the best way to classify a paper isn't to say "This is Biology," but to say "This is mostly Biology, but also a little Chemistry."

Summary

In short, this paper is about building a new, giant map of human knowledge. Instead of just counting how many times scientists cite each other, they used AI to read the actual text of 56 million papers. They found that this creates a beautiful, self-organizing landscape where ideas naturally cluster together. They proved that combining this "reading" map with the traditional "citation" map gives us a much clearer, more nuanced view of how science is structured.

Note: The paper focuses strictly on creating and analyzing this map of existing scientific data. It does not claim to use this for predicting future medical cures, clinical treatments, or specific real-world applications beyond the analysis of the data itself.

Technical Summary: Mapping the Web of Science with LLM Embeddings

Problem Statement

The field of scientometrics faces a fundamental challenge in categorizing sciences and organizing knowledge systems. Traditional approaches rely heavily on citation graphs, where relationships are defined by references and co-citations. While effective for clustering, these graph-based methods are distinct from Natural Language Processing (NLP) techniques that analyze the semantic content of individual texts. A significant gap exists in integrating these two independent methodologies: graph theory abstracts relations between media, while NLP analyzes the text itself. Mathematically, these approaches employ incompatible methods, and attempts to translate between them often result in substantial information loss. Furthermore, existing classification schemas (such as those used by Web of Science) are often top-down and lack the neutrality of bottom-up, data-driven approaches.

Methodology

The authors propose a large-scale investigation of the Web of Science (WoS) dataset using Large Language Model (LLM) embeddings to create a text-based map of science, which is then compared against graph-based structures.

Data Preparation

Dataset: The study utilizes the Web of Science collection up to 2024, comprising approximately 56.1 million records linked by nearly 988 million citations.
Sampling: A random subsample of 48,076 records was selected (probability $10^{-3}$ ). After preprocessing (removing records lacking publication years, authors, references, journals, titles, unique identifiers, or abstracts under 100 characters), the final working set contained 41,901 records.
Hardware: Experiments were conducted on an NVIDIA Grace Hopper architecture, though the authors note the methods are feasible on commodity hardware.

Embedding Strategy

Models: Two open-source LLM embedding models were employed via the ollama framework:
1. mxbai-embed-large:335m (1024 dimensions).
2. nomic-embed-text:v1.5 (768 dimensions).
Rationale: Scientific abstracts were chosen as the input text because they are descriptive, formulaic, homogeneous, and contain relevant keywords, making them suitable for LLM embedding without the linguistic complexities found in full texts.
Metric: Normalized vectors were generated, utilizing cosine similarity as the distance metric. The authors conceptualize the resulting point cloud as lying on an $n$ -sphere, where clustering represents "drawing a map" on this globe.

Analytical Framework

Subject Centers: The center-point for each of the 255 unique WoS subjects was computed as the weighted average of the embedding vectors for records within that subject (weighted by the inverse of the total number of subjects per record).
Distance Correlation: Pairwise distances in the embedding space were compared against shortest-path distances on the citation graph to test for positive correlation.
Dimensionality Reduction: Principal Component Analysis (PCA) was performed to evaluate explained variance across dimensions.
Hybrid Potential: The study investigates the interplay between "interdisciplinary" records (high variance in embedding space, low graph distance) and "distant topic similarity" (low graph distance, high embedding proximity).

Key Results

Embedding Space Characteristics

Full Dimensionality: PCA analysis revealed that both embedding models populate all dimensions of their vector spaces, even with homogeneous text data. The authors conclude that dimensionality reduction may come at the cost of granularity and is not necessarily beneficial for subsequent clustering.
Correlation with Citations: A positive correlation was observed between embedding space distances and citation graph distances.
- Pearson Correlation Coefficient (PCC): 0.455 for mxbai-embed-large:335m.
- PCC: 0.337 for nomic-embed-text:v1.5.
- The correlation is stronger for closer pairs and diminishes for distant pairs.

The Map of Sciences

Structural Formation: The embedding space reveals a self-structured landscape where natural sciences, social sciences, and humanities form large, interconnected "clouds," with their respective branches forming sub-clusters.
Subject Distribution: The distribution of records varies by subject, heavily influenced by interdisciplinary nature.
Soft Classification: Due to multi-labeling (records belonging to multiple subjects) and the observed continuous distributions, the authors assert that scientific classification cannot be rigid. Instead, it must be "soft," assigning probability distributions over multiple subjects rather than discrete categories.

Significance and Claims

The paper does not claim to have solved the classification problem definitively but rather demonstrates the practicability and potential of using LLM embeddings for large-scale scientometric analysis.

Complementarity: The primary significance lies in showing that text-based embeddings and graph-based methods are not mutually exclusive but can be combined. The authors argue that because the two methods are methodologically different, their respective outliers are likely independent.
Hybrid Future: The paper posits that a hybrid approach—combining embedding distance and graph distance—could compensate for the shortcomings of each individual method. This combination promises improved accuracy in topic classification and dataset clustering.
Foundation: The work serves as a foundation for predicting and classifying the location of texts within a semantic landscape, offering a "bottom-up" alternative to existing, potentially biased, taxonomy systems.

The authors explicitly state that future work will investigate the specific implementation of this combined measure for feature selection to achieve higher accuracy, rather than presenting a finalized, superior classification system in this study.