GPS as a Control Signal for Image Generation

We show that the GPS tags within image metadata provide a useful control signal for image generation. We train GPS-to-image models and use them for tasks that require a fine-grained understanding of how images change their appearance throughout a city. In particular, we train a diffusion model to generate images conditioned on both GPS tag and text, and find that the resulting model learns to successfully vary the contents of its generated images using both control signals, such as by creating images that capture the distinctive appearance of different neighborhoods, parks, and landmarks. We also show that GPS conditioning enables us to ``lift'' 3D models from 2D GPS-to-image models using score distillation sampling, without need for explicit camera pose estimation. Our evaluations suggest that our GPS-conditioned models successfully learn to generate images that vary based on location, and that GPS conditioning improves estimated 3D structure.