Unsupervised 3D Neural Rendering of Minecraft Worlds

We present GANcraft, an unsupervised neural rendering framework for generating photorealistic images of large 3D block worlds such as those created in Minecraft. Our method takes a semantic block world as input, where each block is assigned a label such as dirt, grass, tree, sand, or water. We represent the world as a continuous volumetric function and train our model to render view-consistent photorealistic images from arbitrary viewpoints, in the absence of paired ground truth real images for the block world. In addition to camera pose, GANcraft allows user control over both scene semantics and style.


Outputs from our model. The input block worlds are shown as insets.