When fresh data isn’t available, developers usually turn to “derived” data—either recycling the same text over multiple training rounds (multi-epoch repetition) or rewording it using AI (paraphrasing). To measure how well this works, the researchers introduced a concept called token effectiveness ($\eta$). Think of it as a value score: a completely fresh, original word gets a 1.0, while a repeated or reworded token might score lower depending on how useful it remains to the model.

We introduce, a new large-scale dataset and family of generative models for generating Minecraft worlds at block resolution. Our data comprises billions of high-quality and carefully-balanced cubes from procedurally generated Minecraft terrain and human-authored maps, which we use to study discrete and continuous 3D diffusion models for biome-conditioned chunk generation. When trained with our data, we show that both approaches can generate high-fidelity chunks of the game world, and that the discrete masked diffusion formulation gives us inpainting, outpaining, and user-defined block conditioned generation as a free byproduct of the training objective. This enables players and creators to mold the world around them by generating structures, terrain, and maps that are immediately editable and playable.

Why Minecraft?

Generative AI has made incredible progress in the fields of image, video, and text generation. Despite success in these modalities, the interactive 3D worlds of video games have received much less research attention. We aim to close this gap by releasing a dataset based on one of the most successful and popular video games, Minecraft.