From Next Token Prediction to (STRIPS) World Models
This paper investigates whether next-token prediction can learn symbolic STRIPS world models for planning, finding that while a specialized STRIPS Transformer offers theoretical alignment, a standard transformer with stick-breaking attention achieves superior training accuracy and generalization, enabling effective planning across unseen states and goals.