Predictable Emergence: An Empirical Analysis of Whether Sharp Capability Jumps Follow from Smooth Per-Token Scaling Laws
This paper demonstrates that the sharp "emergent" jumps in exact-match accuracy observed in multi-digit integer addition are not genuine discontinuities in model capability but are instead fully predictable artifacts resulting from smooth, power-law improvements in per-token accuracy compounded by the nonlinearity of requiring all digits to be correct.