SyncSDE: A Probabilistic Framework for Diffusion Synchronization

There have been many attempts to leverage multiple diffusion models for collaborative generations, extending beyond the original domain. One prominent approach is synchronizing multiple diffusion trajectories by mixing the estimated scores to artificially correlate the generation processes. However, existing approaches rely on heuristics such as averaging for synchronizing trajectories. Such approaches do not clarify WHY such methods work, and also create many failure cases when the heuristic used on one task is naively applied to other tasks. In this paper, we present a probabilistic framework for analyzing why diffusion synchronization works, and prove that heuristics model the correlations between multiple trajectories, hence must be adapted accordingly to each task the synchronization takes place. We attempt to find the best correlation models for each tasks, giving the best results compared to previous approaches which naively applies single heuristics to every tasks without reasoning.