We present an efficient diffusion-based method for video frame interpolation. Most recent diffusion-based methods still have a large gap from non-diffusion methods in accuracy and efficiency. The key of our method is, instead of formulating the problem as a denoising procedure in the latent space directly, which is less effective caused by the large latent space, we propose to model optical flow explicitly from coarse to fine by a hierarchical diffusion models, which has much smaller search space in each denoising step, and can handle complex motions and large displacements. Extensive evaluation on multiple benchmarks demonstrates that our method achieves state of the art in accuracy, and 10+ times faster than other diffusion-based methods.