💻 computer science

Attention-Augmented LSTMs for Automatic Homophonic Ciphertext Decipherment

This paper demonstrates that an attention-augmented LSTM model, trained solely on aligned ciphertext-plaintext pairs without external linguistic resources, can achieve near-perfect automatic decipherment of historically motivated homophonic substitution ciphers by learning shared code pools across varying languages, time periods, and noise levels.

Original authors: Micaella Bruton, Meriem Beloucif, Beáta Megyesi

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Micaella Bruton, Meriem Beloucif, Beáta Megyesi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Cracking a Secret Code with a "Super-Reader"

Imagine you are trying to read a secret diary from the 1700s. But there's a catch: the writer didn't just swap the letter "A" for the number "1." Instead, they had a huge bag of numbers (a "pool") and could pick any number from that bag to represent an "A." Sometimes they used "1," sometimes "42," and sometimes "999."

This is called a Homophonic Cipher. It's a clever trick designed to confuse code-breakers because the usual trick of counting how often a letter appears (frequency analysis) doesn't work. If "A" can be 1, 42, or 999, you can't tell which one is "A" just by looking at the numbers.

This paper asks a simple question: Can a modern computer program (specifically an AI called an LSTM with "attention") learn to crack these codes automatically, even without a human telling it the rules?

The Setup: A Shared "Bag of Tricks"

The researchers didn't just test one secret code. They created a scenario that mimics real history:

The Shared Pool: Imagine a giant library of all possible secret numbers (the "key space").
The Individual Keys: Different writers (or different letters) only use a subset of that library. One writer might use numbers 1–100 for "A," while another uses 50–150.
The Rule: Crucially, within any single document, if the number "42" appears, it always means the same letter (e.g., "A"). It never changes to mean "B" in the same document.

The researchers trained their AI on thousands of these fake secret messages generated from historical English and Swedish texts (from 1500 to 1899). They gave the AI the secret code and the real message, but no dictionaries, no grammar rules, and no human hints. The AI had to figure out the pattern on its own.

The "Attention" Superpower

The AI they used is an LSTM (a type of neural network good at reading sequences) with a special add-on called Attention.

The Analogy: Think of reading a long, confusing sentence where a word is missing. You might look back at the beginning of the sentence or forward to the end to guess what the missing word is.
The AI's Job: The "Attention" mechanism lets the AI look at the whole secret message at once. If it sees a weird number, it can look at the surrounding numbers to figure out, "Ah, in this specific context, this number must be an 'E' because it's surrounded by numbers that usually make 'THE'."

The Results: Almost Perfect

The researchers tested the AI under very difficult conditions:

Short Messages: Only 50 characters long (very little context).
Old Languages: Texts from 500 years ago with old spelling.
Messy Data: Simulated "typos" (like a human transcribing a handwritten note and accidentally writing the wrong number).
Variable Lengths: Some codes were 3 digits, others were 4 digits.

The Outcome:
The AI was incredibly successful.

Accuracy: It got the decryption right almost 100% of the time for clean messages.
Noise: Even when the messages had "typos" or mixed code lengths, it still got it right over 99% of the time.
Time Travel: It worked just as well on 1500s text as it did on 1800s text. It didn't need to be retrained for different centuries.

The "Magic Trick": Knowing When It Doesn't Know

One of the most interesting findings was what happened when the researchers gave the AI a secret message that did not use the same "bag of numbers" (the shared pool) it was trained on.

The Result: The AI failed immediately and predictably. It didn't guess randomly; it just couldn't solve it.
Why this matters: This proves the AI didn't just memorize the specific messages it studied. It actually learned the structure of the shared code pool.
The Analogy: It's like a person who learns to recognize a specific brand of car engine. If you show them that engine, they can fix it. If you show them a completely different brand, they say, "I don't know this engine," rather than trying to force a wrench into the wrong place. This makes the AI a useful tool for historians to check: "Does this new, mysterious letter use the same secret code as the ones we already know?"

The "Glitch" in the System

When the AI did make a mistake, it wasn't usually because it mixed up letters (like thinking "A" was "B").

The Real Issue: The mistakes usually happened because the AI got confused by the "typos" (transcription errors). It would correctly identify the secret code but get confused about where the typo was.
The Takeaway: The underlying logic of the code was still intact; the AI just stumbled over the messy handwriting simulation.

Summary

This paper shows that a specific type of AI can learn to crack complex, historical secret codes without needing a human to teach it the rules of English or Swedish.

It works on short, long, old, and messy texts.
It learns the "shared rules" of the code so well that it can tell you if a new message follows those same rules.
It acts as a powerful assistant for historians, helping them verify if a mysterious document belongs to a known group of secret letters, even if the handwriting is messy or the text is very short.

Technical Summary: Attention-Augmented LSTMs for Automatic Homophonic Ciphertext Decipherment

Problem Statement
Homophonic substitution ciphers present a significant challenge for automated cryptanalysis because they map a single plaintext character to multiple possible ciphertext codes. This design deliberately obscures letter-frequency distributions, rendering standard frequency-based analyses ineffective. Traditional automated approaches rely on heuristic search methods (e.g., nested hill climbing, simulated annealing, beam search) combined with language-specific fitness functions (n-gram statistics). While successful on specific historical ciphers, these methods often require extensive linguistic resources and struggle with noisy or irregular inputs. Furthermore, recent neural sequence models applied to cipher decryption have largely been tested on clean, modern data and simple cipher types (e.g., Caesar, Vigenère), leaving their efficacy on historically motivated homophonic ciphers underexplored.

This paper investigates whether a neural sequence model, specifically an attention-augmented Long Short-Term Memory (LSTM), can serve as a practical tool for deciphering homophonic ciphers under historically realistic conditions. The study focuses on a "shared-key space" setting: a scenario where all ciphertexts in a dataset draw from the same known pool of homophonic codes, though individual keys utilize different consistent subsets of that pool. This mirrors real-world historical scenarios, such as the Mary Stuart letters, where subsets of a larger key space are reused.

Methodology
The authors formulate the decryption task as a supervised sequence-to-sequence prediction problem. Given a ciphertext sequence $C$ , the model predicts the corresponding plaintext sequence $P$ at the character level.

Dataset: The study utilizes a synthetic dataset generated via ChronoFidelius from historical English and Swedish texts (1500–1899). The data spans six text lengths (50 to 1,000 characters) and is temporally balanced across centuries.
Experimental Variants: To simulate archival challenges, four ciphertext variants were created for each plaintext:
1. Clean ciphertext.
2. Ciphertext with 5% random character-level substitutions/insertions (simulating transcription errors).
3. Ciphertext with variable-length codes (3-digit for vowels, 4-digit for consonants).
4. Ciphertext combining both noise types.
Model Architecture: The proposed model is a bidirectional LSTM (biLSTM) augmented with a multi-head self-attention mechanism.
- Ciphertext symbols are embedded and processed by a multi-layer biLSTM to capture left and right context dependencies.
- A multi-head attention mechanism operates over the full sequence of hidden states, dynamically reweighting relevant positions to resolve homophonic ambiguity.
- The attention output is concatenated with the LSTM hidden state, normalized, and passed to a linear projection layer to output a probability distribution over the plaintext vocabulary.
Training Constraints: Crucially, the models are trained only on aligned ciphertext–plaintext pairs. No external language models, frequency statistics, or key-search heuristics are used. The goal is to determine if the model can learn the decryption mapping solely from the structural relationship between the ciphertext and plaintext within the shared key space.
Evaluation: Performance is measured using character-level F1 scores. Experiments include diachronic evaluation (testing on specific centuries not seen in training) and 5-fold cross-validation on short (50-character) ciphertexts to assess robustness and prevent overfitting.

Key Results
The study reports near-perfect character-level decryption accuracy across both languages, all historical periods, and varying ciphertext lengths.

Clean and Variable-Length Data: Models achieved an F1 score of approximately 1.0 for clean ciphertexts and those with variable-length codes, regardless of text length.
Noise Resilience: When transcription errors were introduced, performance remained extremely high (F1 > 0.99). The combination of transcription errors and variable-length codes resulted in a minor performance decrease (F1 > 0.986), demonstrating strong robustness to realistic noise.
Diachronic Generalization: Performance remained stable across all tested centuries (1500–1899), indicating the model generalizes across historical language variations when trained on temporally balanced data.
Short Ciphertexts: 5-fold cross-validation on 50-character sequences confirmed that the model maintains near-perfect accuracy (mean F1 $\ge$ 0.999) even with limited context, with zero variance in several configurations.
Key-Space Verification: When tested on ciphertexts encrypted with keys drawn from outside the shared homophonic pool, the model's performance collapsed (F1 $\le$ 0.08). This predictable failure confirms the model learns the specific shared key space rather than superficial patterns.

Error Analysis
Inspection of errors reveals that failures are not random substitutions. Instead, they predominantly arise from:

Misidentifying the presence or position of transcription errors (e.g., failing to flag an <ERR> token).
Systematic misidentification of specific homophonic codes (e.g., consistently decoding a specific code as 'I' instead of 'C').
The underlying substitution structure is largely preserved even in error cases, suggesting the model learns the mapping logic rather than guessing.

Significance and Claims
The paper claims that attention-augmented LSTMs can function as a robust, practical tool for historical cryptanalysis within a shared key space setting. Key contributions include:

Independence from Linguistic Resources: The model achieves high performance without relying on external language models or n-gram statistics, learning decryption behavior directly from the ciphertext-plaintext alignment.
Robustness to Historical Realities: The approach effectively handles diachronic language variation, variable-length codes, and simulated transcription errors, addressing gaps in prior neural work which often relied on clean, modern data.
Key-Space Verification Tool: The model's predictable failure on ciphertexts outside the known key space allows it to serve as a verification tool. Researchers can use it to scan unknown ciphertexts to determine if they share a known key space, even if full decipherment is not immediately possible.
Complementary Role: The authors position this method not as a replacement for established cryptanalytic expertise, but as an assistive tool that complements traditional heuristic search methods, particularly for large-scale or heterogeneous archival collections.

The study concludes that while the current approach is limited to shared key spaces, it establishes a solid foundation for using neural sequence models in historical cryptanalysis, with future work targeting independent key space decryption and end-to-end pipelines integrating transcription and decipherment.