🤖 AI

MUSEKG: A Knowledge Graph Over Museum Collections

MuseKG is an interactive knowledge graph system that unifies fragmented museum collection data into a coherent, queryable schema, enabling natural-language exploration and relation-aware retrieval of objects, people, and semantic entities with inspectable evidence.

Original authors: Jinhao Li, Jianzhong Qi, Soyeon Caren Han, Eun-Jung Holden

Published 2026-05-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jinhao Li, Jianzhong Qi, Soyeon Caren Han, Eun-Jung Holden

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, ancient library where the books (museum artifacts) are scattered across different rooms. Some books are in a dusty filing cabinet with handwritten cards, others are in a digital database with just a photo and a title, and some are in a separate room with loose notes about who made them. Trying to find a specific book, understand who wrote it, or see how it connects to other books is like trying to solve a puzzle with missing pieces.

This is the problem MuseKG solves. It acts like a super-smart librarian who takes all those scattered, messy pieces of information and organizes them into a single, giant, interactive web of connections.

Here is how the paper explains it, using simple analogies:

1. The Problem: The "Jigsaw Puzzle" of Museums

Museums have digitized millions of items, but the data is "fragmented." It's like having a jigsaw puzzle where the pieces are in different boxes, and some pieces are just descriptions while others are just pictures. You can't easily see the whole picture or ask questions like, "Show me all the metal tools made by this specific company in 1920."

2. The Solution: The "Living Web" (The Knowledge Graph)

The authors built MuseKG, which turns those scattered records into a Knowledge Graph.

The Analogy: Imagine a giant spiderweb.
- The dots on the web are the things: the artifacts (like a "Long Scale Galvanometer"), the people who made them, the organizations, the photos, and even words found in the descriptions (like "aluminum" or "kitchen appliance").
- The strings connecting the dots are the relationships: "Made by," "Is a type of," "Has a photo of," or "Contains."
What it does: Instead of just listing items, MuseKG builds a map that shows how everything is connected. If you click on a "Galvanometer," the web instantly shows you the person who built it, the photo of it, and the specific materials used, all linked together.

3. How You Talk to It: The "Smart Translator"

You don't need to know computer code (like SPARQL) to use this. You just talk to it in normal English.

The Process:
1. You ask: "Who made the Long Scale Galvanometer?"
2. The System listens: It uses a smart AI (an LLM) to find the "Galvanometer" in the web.
3. It follows the string: It traces the "Made by" string to the "Walden Precision Apparatus Limited" dot.
4. It answers: It tells you the name of the company.
The Safety Net: Unlike some AI that just guesses or "hallucinates" facts, MuseKG is strict. It only answers using the facts it found on the web. If the answer isn't on the web, it won't make one up. It shows you the "evidence" (the specific part of the web it looked at) so you can trust the answer.

4. What You Can Do (The Demo)

The paper shows three main ways people can use this system:

The Detective (Attribute Lookup): You ask, "How big is this object?" and the system pulls up the exact measurements from the catalog.
The Explorer (Relation Exploration): You ask, "Who made this?" and the system shows you the maker and lets you click on them to see what else they made.
The Investigator (Relation-Aware Retrieval): You ask a two-step question: "What is the ID number of the company that made this?" The system follows the path: Object -> Maker -> ID Number and gives you the answer, showing you the path it took.

5. Did It Work? (The Test)

The authors tested MuseKG against other methods.

The Result: When asked simple questions, standard AI sometimes guessed wrong. When asked complex questions involving connections (like "Who made X and what is their ID?"), standard AI struggled a lot.
MuseKG's Win: Because MuseKG forces the AI to look at the actual "web" of facts before answering, it got the right answers much more often, especially for questions that required following connections between items.

Summary

MuseKG is a tool that takes a museum's messy, scattered digital records and weaves them into a single, connected map. It lets you ask questions in plain English, finds the answers by following the connections on that map, and shows you the proof so you know the answer is real. It turns a static database into an interactive conversation with the museum's history.

Technical Summary: MuseKG – An Interactive Knowledge Graph Over Museum Collections

Problem Statement

The cultural heritage sector has generated vast repositories of museum collection data through digitization efforts. However, these repositories are often fragmented, consisting of heterogeneous sources such as structured catalogue records, image archives, and unstructured descriptions. Existing museum information systems struggle to integrate these sources into a unified, queryable representation that supports relation-aware exploration. The primary obstacles are not merely the scale of data but the semantic disconnection between object descriptions, relationships, and contextual metadata. While recent advances in Large Language Models (LLMs) and Knowledge Graphs (KGs) offer potential solutions, existing approaches often fail to bridge the gap between end-to-end pipelines. Specifically, standard-centric efforts assume expert SPARQL usage, while automated construction methods often lack explainable query experiences for end users. Furthermore, current NL2SPARQL techniques can be brittle under complex schemas, and interactive KG tools are rarely integrated with multimodal museum artifacts.

Methodology

The authors present MuseKG, an interactive system designed to organize heterogeneous museum data into a typed property graph and provide a natural-language (NL) interface for querying and exploration.

1. System Architecture

MuseKG operates through two core modules:

Module 1: KG Constructor: This component transforms raw museum collection records (JSON format) into a typed property graph $G = (V, E, \tau, \rho, A)$ .
- Data Processing: The system ingests records containing object metadata (titles, materials, dates) and relational fields (people, organizations, related objects, image labels).
- Construction Steps:
  1. Normalization: Textual fields are standardized (lowercased, whitespace collapsed) to aid entity resolution.
  2. Entity Identification: Nodes are created for object IDs, referenced entities (persons, organizations), image IDs, and named entities extracted from text.
  3. Type Assignment: Nodes are assigned types (e.g., object, person, image, semantic entity).
  4. Relation Labelling: Raw relationship identifiers are mapped to a fixed vocabulary of seven relation types across four categories: provenance, inter-object, visual, and semantic.
  5. Deduplication: Duplicate nodes and edges are merged or removed based on canonical identifiers and triples.
  6. Schema Validation: Attributes are restricted to a predefined schema.
- Output: The constructed graph links objects, people, organizations, images, image-derived labels, and semantic entities. In the demonstration, this resulted in a graph with 23,263 nodes and 75,375 edges derived from 15,829 object records.
Module 2: Natural Language Query Interface: This module utilizes a Retrieval-Augmented Generation (RAG) approach to answer user queries.
- Context Retrieval: Upon receiving an NL query, an LLM extracts key entities. The system then retrieves a textual context from the KG, comprising the attributes of the identified entity and its one-hop neighbors (relations and connected nodes).
- Answer Generation: The retrieved context and the original query are fed into an LLM, which is prompted to synthesize a concise answer using only the provided KG evidence. This design aims to mitigate hallucinations common in LLM-only systems.

2. Interaction Workflow

The system provides a visual interface where users can type free-form questions or select prompts. The system:

Grounds the query to specific entity mentions.
Retrieves a compact KG neighborhood as evidence.
Visualizes the subgraph for inspection.
Displays an entity detail card with metadata and images.
Generates a natural-language answer grounded in the retrieved facts.

Key Contributions

The paper outlines three primary contributions:

MuseKG System: An interactive KG system that enables the construction and querying of museum collections via natural language, integrating structured records with media-derived labels.
Lightweight Pipeline & Interface: A streamlined KG construction pipeline and NL query interface supporting three specific tasks: attribute lookup, relation exploration, and relation-aware retrieval.
Demonstration & Benchmark: A demonstration on real museum collections showing interpretable, evidence-grounded answers, supported by a benchmark of 150 questions generated via KG-grounded and LLM-guided methods.

Results and Evaluation

The authors evaluated MuseKG against several baselines using a custom benchmark of 150 questions categorized into:

C1: Attribute lookup (e.g., retrieving measurements).
C2: Relation exploration (retrieving linked entities like producers).
C3: Relation-aware attribute retrieval (traversing a relation to find an attribute of the connected entity).

Comparative Performance:
MuseKG was compared against LLM-only baselines (Zero-shot, Few-shot) and an NL2SPARQL prompting pipeline across three different backbone LLMs (Gemma-3-4B, Gemma-3-12B, and gpt-oss-20b).

Accuracy: MuseKG consistently achieved the highest accuracy across all categories and model sizes. For instance, on the Gemma-3-12B model, MuseKG achieved 0.84 accuracy on C1, 0.42 on C2, and 0.36 on C3.
Baseline Comparison: While the NL2SPARQL baseline was competitive on attribute lookup (C1), it performed significantly weaker on relation-centric tasks (C2 and C3). MuseKG's explicit KG-grounded execution provided the largest gains in these complex relation traversal scenarios.
Prompting: Unlike the baselines which required category-specific prompt engineering (few-shot or SPARQL-prompting), MuseKG utilized a single generic KG-RAG prompt while maintaining superior performance.

Significance and Claims

The paper claims that MuseKG successfully bridges the gap between fragmented museum records and accessible natural language querying. Its significance lies in demonstrating how combining the structural precision of a knowledge graph with the flexibility of LLMs can provide inspectable, evidence-grounded answers.

The authors emphasize that the system allows users to verify the provenance of answers by inspecting the explicit graph structures and relations used to generate them. They note that MuseKG outperforms baselines specifically on relation-centric tasks, addressing the brittleness of direct NL-to-SPARQL generation in complex schemas.

The paper maintains a modest scope, acknowledging current limitations: the system currently resolves only a single main entity per query, limiting its ability to handle comparative questions across multiple entities or open-ended questions without specific object references. Future work is proposed to scale the pipeline for cross-collection discovery and evolve the system into an agent-based framework for autonomous investigation across heterogeneous networks.