Era-Aware Language Modeling: Training a Decoder-Only Transformer on a Balanced Gutenberg Corpus
This paper introduces GutenbergLM, a 109.5M-parameter decoder-only transformer trained from scratch on a temporally balanced Project Gutenberg corpus with era-conditioning tokens, demonstrating that explicit temporal stratification enables the model to generate distinct, period-appropriate writing styles across Early, Middle, and Modern literary eras.