💻 computer science

Calibrated evidence fusion recovers local reading order in 4,251 excavated Chu bamboo slips

This paper introduces a calibrated multi-evidence fusion method, incorporating a novel language-model junction score, to objectively rank candidate neighbors for reassembling fragmented Chu bamboo slips, significantly outperforming chance in continuous texts while demonstrating that complete ordering still requires physical numbering.

Original authors: Weishan Yang, Jie Yang

Published 2026-09-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Weishan Yang, Jie Yang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a library where the books have been shredded into thousands of tiny strips of wood, and the strings that once held them together have long since turned to dust. This is the reality for archaeologists working with ancient Chinese bamboo manuscripts. When these texts are dug up, they arrive as a chaotic pile of fragments. Scholars have spent centuries trying to piece them back together, a process that involves two distinct challenges. First, they must decide which strips belong to the same book, a task often solved by matching physical features like the length of the wood or the position of notches cut into the sides. Second, and far more difficult, is figuring out the correct reading order of the strips once they are identified as belonging to one volume. Traditionally, this second step has relied heavily on the human eye and mind, with experts reading the text and guessing which sentence flows naturally into the next. This method is slow, subjective, and prone to error, leaving many ancient stories and records in a state of permanent disarray.

A new study by researchers Weishan Yang and Jie Yang offers a different approach, using computer science to bring clarity to this chaos. They developed a method that treats the problem like a complex puzzle, where the goal is to find the single best path that connects all the pieces in the right sequence. Instead of relying solely on a human's intuition, their system analyzes the text on every slip, looking for patterns in vocabulary and sentence structure that suggest one piece should follow another. They tested this system on a massive collection of 4,251 excavated bamboo slips from the Chu state, covering everything from administrative records and daily logs to philosophical treatises and divination archives. The researchers did not just ask the computer to guess the whole order at once; instead, they asked it to provide a short list of the most likely neighbors for each individual slip, a task that mirrors how a human editor actually works through a fragmented manuscript.

The results show that the computer can act as a powerful assistant, narrowing down the possibilities dramatically. For slips containing continuous prose, such as philosophical essays or historical chronicles, the system correctly identified the true next piece as the top candidate in many cases, and placed it within the top five choices for about half to two-thirds of all the slips. This is a significant improvement over random guessing, which would require an editor to sift through dozens of unrelated pieces to find the right match. The study found that the type of text matters greatly; documents that tell a continuous story or argument are much easier to order than lists of inventory or records of divination, where entries often stand alone and do not flow into one another. In fact, for the latter types of records, the computer's ability to predict the next piece was barely better than chance, highlighting that the text itself sometimes holds no clues about the sequence.

To make the system even smarter, the researchers added a new layer of analysis that looks at the very edge where two slips meet. They used a language model trained on classical Chinese to check if the end of one sentence flows smoothly into the beginning of the next, a technique that helps determine not just that two pieces belong together, but which one comes first. This directional clue proved very effective for continuous texts, helping the system distinguish between similar-looking pieces. However, the study also revealed a hard limit to what text alone can achieve. While the computer could reliably find the immediate neighbors of a slip, it could not always assemble the entire manuscript in the correct order from start to finish. The researchers found that without physical clues like numbers written on the slips or their original position in the ground, the computer often gets stuck in local loops, finding correct pairs but failing to link them into a single, unbroken chain.

The team also ran simulations to test what would happen if they had access to physical measurements that are often missing from digital records, such as the exact length of each slip or the position of its notches. In these controlled tests, adding physical data helped the system perform much better, confirming that while text is excellent for finding local connections, physical evidence is likely required to solve the global puzzle of the entire book's order. The study concludes that while we cannot yet fully reconstruct these ancient libraries with a single click, the new method provides a reliable way to generate a shortlist of candidates for every single slip. This reduces the work for human scholars from reading tens of potential matches down to just a handful, turning a nearly impossible task into a manageable one. The path forward involves training computers to read the strict, unaltered handwriting of the original slips and making physical measurements machine-readable, but for now, this calibrated fusion of text and logic offers the most precise tool yet for recovering the lost voices of the Chu bamboo slips.

Technical Summary: Calibrated Evidence Fusion for Recovering Local Reading Order in Chu Bamboo Slips

Problem Statement
The reconstruction of excavated bamboo manuscripts involves two distinct tasks: categorization (determining which fragments belong to the same manuscript) and sequencing (recovering the original binding order). While machine learning has advanced categorization and fragment joining, sequencing remains a significant bottleneck. Current scholarship relies heavily on subjective textual coherence once physical clues (excavation position, numbering) are exhausted. The authors argue that sequencing is a seriation problem where the goal is to find a constrained maximum-weight Hamiltonian path over slips assigned to a single manuscript. The core challenge is that existing published machine-readable corpora lack physical measurements (slip length, notch position, stratigraphic level), forcing reliance on textual evidence alone, which is often insufficient for establishing a complete global order.

Methodology
The study proposes a calibrated, reproducible evidence fusion framework solved via constraint programming (CP-SAT). The methodology proceeds as follows:

Evidence Channels: The system integrates three primary types of evidence, normalized via z-scoring:
- Text Similarity: Uses 1–2 character n-gram TF-IDF vectors and cosine similarity to identify slips sharing vocabulary, names, or subject matter. This is symmetric (direction-agnostic).
- Junction Score (Directional): A novel channel using a BERT model pretrained on Classical Chinese. It calculates the probability gain of the first characters of slip $B$ given the last characters of slip $A$ (masked language modeling) versus their probability within their own slips. This provides a directional signal for continuous prose.
- Physical Evidence: In the synthetic benchmark, this includes slip length, notch positions, stratigraphic levels, and ancient numbering. These are noted as categorization or ordering constraints but are absent from the real-world corpora tested.
Fusion and Calibration: A logistic regression model, trained on a held-out calibration corpus, learns weights for these channels to predict whether a pair of slips is truly adjacent. Channels with no data for a specific pair contribute zero weight.
Optimization: The problem is formulated as a constrained maximum-weight Hamiltonian path.
- A dummy node connects to all slips with zero cost to transform the open path into a circuit solvable by CP-SAT.
- Candidate arcs are restricted to the top 14 nearest neighbors (greedy chain) to reduce search space.
- Hard constraints (e.g., known first/last slips) and soft constraints (stratigraphic windows) are applied.
- The solver outputs either a global order or, more practically, a ranked shortlist of candidate neighbors for each slip.
Evaluation:
- Retrieval Measure: For each slip, checks if a true neighbor (immediate predecessor or successor) appears in the top $K$ candidates.
- Strict Measure: Evaluates the global order using Kendall $\tau$ , adjacent-pair accuracy, and longest correct run.
- Corpora: Tested on 4,251 slips across three corpora: Baoshan (administrative/divination), Guodian (philosophical treatises), and CHUBS (character images from Tsinghua/Shanghai Museum collections). A synthetic corpus with full ground truth was used to test physical evidence channels.

Key Results

Local Adjacency Recovery: The method successfully recovers local neighbors. For continuous texts, the true neighbor is the top candidate 6 to 25 times more often than chance. In continuous prose, the true neighbor falls within the top 5 candidates for 50% to two-thirds of slips.
Genre Dependence: Performance varies significantly by genre. Administrative documents, chronicles, and day-books perform well. Inventories and divination archives perform near chance levels because their entries are templated or near-duplicates, lacking the continuous syntactic flow required for the junction score to function.
Junction Score Efficacy: The language-model junction score adds a calibrated gain specifically for continuous prose (e.g., +0.046 recall on treatises) but offers no benefit for archives or inventories. It is the first directional channel tested on excavated material.
Global Order Limitations: The strict measure (Kendall $\tau$ ) for complete manuscript reconstruction remains low (e.g., $\tau \approx 0.13$ for CHUBS documents), indistinguishable from random in some cases (e.g., the Laozi collection). Textual coherence alone cannot arbitrate between locally plausible continuations to fix a whole chain.
Synthetic Findings: On synthetic data with full ground truth, physical evidence (numbering) provides a "floor" for performance ( $\tau \approx 0.4$ ) even when text is noisy, whereas text-only methods collapse. Notch position and length were confirmed as categorization tools rather than sequencing tools.

Significance and Claims
The paper claims a modest but practical advancement in computational philology:

Tool for Editors: The primary contribution is a candidate-ranking tool that reduces the editorial workload. By narrowing the search space from tens of candidates to a shortlist of five, it cuts the reading required per slip by an order of magnitude.
Clarification of Limits: The study precisely defines the boundary of what textual coherence can achieve. It confirms that while coherence is a reliable local witness, it is an unreliable global witness. The gap between local recovery and global ordering is bridged by physical clues (numbering, anchors) which are currently missing from machine-readable datasets.
Methodological Validation: The work validates that a calibrated fusion of textual similarity and a directional language-model score, solved via constraint programming, is the optimal approach for the available data. It also identifies that current models struggle with strict palaeographic transcriptions, suggesting a need for models trained specifically on such data.

The authors conclude that while a fully automated reconstruction of entire manuscripts is not yet achievable with current public data, the "local reading order" can be significantly assisted, providing a concrete, reproducible step forward for the field.

Technical Summary: Calibrated Evidence Fusion for Recovering Local Reading Order in Chu Bamboo Slips

More like this