GPTKB 2.0: Browsing, Querying, and Auditing a Disambiguated LLM-Derived Knowledge Base

Imagine the internet as a giant, chaotic library where every book is written by a different author, and sometimes two books have the exact same title but tell completely different stories. In the world of artificial intelligence, "Large Language Models" (LLMs) are like super-smart librarians who have read almost every book in this library. They can answer questions, write stories, and solve problems because they have memorized the patterns of all that text. However, there's a catch: these AI librarians don't always know which "Munich" you are talking about—the famous German city or the 2005 movie. They also might treat "New York" and "The Big Apple" as two totally different places, even though they are the same. This confusion is called the "disambiguation" problem. For years, scientists have been trying to turn the AI's messy, internal memory into a clean, organized map of facts, but it's been like trying to sort a pile of mixed-up puzzle pieces without a picture on the box.

Now, a team of researchers has built a new tool called GPTKB 2.0 to solve this sorting problem. Think of it as a magical, self-correcting encyclopedia that the AI built for itself, but with a twist: it doesn't just list facts; it keeps a detailed diary of how it decided what is true and what is a mistake. Instead of just saying "Munich is a city," this new system asks, "Which Munich? The one in Europe or the one in a movie?" and then creates a separate, clear entry for each one. It also realizes that "The Big Apple" and "New York City" are the same thing and merges them into one perfect entry. The result is a massive, organized database containing over 38 million facts about 1.6 million unique things. But the real magic isn't just the size; it's the transparency. The researchers created a website where anyone can look inside the AI's brain, see the exact questions it asked itself, watch how it decided to merge or split facts, and even ask it questions in plain English. It turns the AI's hidden knowledge into a public, inspectable treasure map that you can explore, query, and trust because you can see exactly how every single fact was discovered.

The Big Idea: A Self-Correcting AI Encyclopedia

The core of this paper is the creation of GPTKB 2.0, a massive knowledge base (a structured collection of facts) built entirely from the "brain" of a Large Language Model. Unlike older attempts that just grabbed words as they appeared (which led to confusion between homonyms like "Munich the city" and "Munich the movie"), this new system performs disambiguation on the fly. It acts like a detective that, every time it finds a new fact, asks, "Who is this really?" and checks the context before writing it down.

The paper presents a web demo that makes this process fully visible. You can browse the database, follow links between facts, and—most importantly—audit the "provenance" of every single fact. This means you can see the surface words the AI saw, the candidate matches it considered, and the specific decision it made to either merge two names (synonymy) or keep them separate (homonymy).

How It Works: The Recursive Detective

The construction of GPTKB 2.0 is a recursive, context-guided pipeline. Imagine the AI starting with a single seed entity, like "Budapest." It asks the LLM for facts about Budapest. When the LLM replies with a new object, say "Munich," the system doesn't just write "Munich." It looks at the sentence: "Budapest has a sister city, Munich." It knows this is the city, not the movie.

The process involves four main steps:

1. Elicitation: The AI asks the LLM for facts based on a specific entity and its description.
2. Named Entity Recognition (NER): It figures out if a new word is a literal number or a real-world entity.
3. Disambiguation: This is the star of the show. The system compares the new mention against existing entries in the database. If the context matches an existing "Budapest," it links to it. If it's a new "Munich" (the movie), it creates a new ID.
4. Consolidation: It merges synonymous names (like "The Big Apple" and "New York") into a single canonical entry.

The paper explicitly argues against the old method of using "surface strings" (just the name) as identifiers. They show that without context, surface strings fail in two ways: they conflate homonyms (treating different things as the same) and fragment synonyms (treating the same thing as different). GPTKB 2.0 rejects this by using the "eliciting triple" (the sentence the fact came from) and the description of existing entities as context to make the right call.

The Numbers: A Massive, Clean Map

The scale of this project is staggering. The paper reports that GPTKB 2.0 contains:

• 38.4 million triples (facts in the form of Subject-Predicate-Object).
• 1.6 million canonicalized entities (unique, disambiguated things).
• 207,633 consolidated relations (types of connections).
• 66,523 consolidated classes (categories of things).

Interestingly, about 36.8% of these entities are new to the world compared to existing databases like Wikidata, meaning the LLM discovered facts that weren't previously in structured databases.

The Web Interface: Transparency is the Feature

The most unique part of this paper is the web interface at https://gptkb.org/. It's not just a search engine; it's a "glass box" into the AI's reasoning.

• Traceable Decisions: If you click on an entity like "Hyde Park," you can see exactly how the system decided it was the one in London and not another one. You can see the list of candidates the AI considered and why it rejected them.
• Synonym Merging: If you click on "The Big Apple," the interface shows you that it was merged into "New York City." You can even click to see the specific prompt the AI used to make that decision.
• Entity Linking: You can type your own text, and the system will link the names in your text to the correct entries in GPTKB 2.0. For example, if you write about "Mercury," it will correctly link the planet to the planet and the Roman god to the god, based on the context of your sentence.
• Querying: You can ask questions in plain English (like "What is Budapest known for?"), and the system translates that into a structured database query (SPARQL) to get the answer. It shows you the exact query it ran and the steps it took.

How Sure Are We? (Evaluation)

The authors didn't just build it; they tested it rigorously. They manually checked samples of the data to see if the AI made mistakes.

• Merge Precision: When the system decided two names were the same thing (synonyms), it was correct 98% of the time for same-label objects and 91% for differently-labeled aliases. The main error mode was false merges (thinking two different things were the same), which happened about 9% of the time in specific alias cases.
• Split Precision: When the system decided two things were different (homonyms), it was 100% correct on the sample of same-label pairs it kept separate.
• Factuality: When checking if the facts were true, 94.5% of the human-checked triples were "true," and only 2.0% were "false." When an AI judge checked 1,000 items, 92.8% were true.

The paper notes that the context (the description of the entity) is crucial. In an experiment where the AI judge didn't see the entity's description, the accuracy of the facts dropped significantly (from 94.5% true to 80% true), proving that context is what makes the disambiguation work.

What It's Not

It is important to note what this paper does not claim. It does not say that GPTKB 2.0 is perfect or that it has solved the problem of AI hallucinations entirely. The authors acknowledge that false synonym merges (merging two different things because they look similar) are the largest remaining error. They also clarify that this is a "general-domain" knowledge base, meaning it covers a wide range of topics but isn't a specialized medical or legal database. The results are based on manual and automatic sampling, not a check of every single one of the 38 million facts, which would be impossible.

The Takeaway

GPTKB 2.0 is a significant step forward because it turns the "black box" of an LLM's knowledge into a "glass box." By building a database that disambiguates entities as it goes and records every decision, the researchers have created a resource that is not only huge and useful but also auditable. You can trust the facts because you can see the evidence behind them. It suggests that we can build structured, reliable knowledge from the chaotic, unstructured knowledge of AI models, provided we are willing to do the hard work of checking the context and keeping a detailed diary of our decisions.