We propose a new object-centric video prediction algorithm based on the deep latent particle (DLP) representation. In comparison to existing slot- or patch-based representations, DLPs model the scene using a set of keypoints with learned parameters for properties such as position and size, and are both efficient and interpretable. Our method, deep dynamic latent particles (DDLP), yields state-of-the-art object-centric video prediction results on several challenging datasets. The interpretable nature of DDLP allows us to perform ``what-if'' generation -- predict the consequence of changing properties of objects in the initial frames, and DLP's compact structure enables efficient diffusion-based unconditional video generation.

Citation
@article{
	daniel2024ddlp,
	title={DDLP: Unsupervised Object-centric Video Prediction with Deep Dynamic Latent Particles},
	author={Tal Daniel and Aviv Tamar},
	journal={Transactions on Machine Learning Research},
	issn={2835-8856},
	year={2024},
	url={https://openreview.net/forum?id=Wqn8zirthg},
	}
}

Code, examples and pre-trained models are available on GitHub.