Towards Data-Efficient and Real-Time World Models
Under Review
A two-stage discrete diffusion-based approach to world modelling: replacing the next-token prediction paradigm with a lightweight refinement transformer that carries out iterative masked predictions. Achieves a 79x speed-up over an autoregressive world model with limited deterioration in output quality, and can be trained to reproduce high-quality, consistent gameplay from roughly one day (28 hours) of gameplay data.