Training a video predictor with a least-square criterion yields blurry frames that average all possible futures.

TRUE: the machine-learning literature as of 2019 establishes that video predictors trained with a squared-error (L2/least-squares) loss produce blurry predicted frames when the future is uncertain, because the loss-minimizing prediction is the conditional mean (an average) of the possible futures. MOSTLY TRUE: blurriness from L2 training is well established, but the 'average of all possible futures' explanation is a simplification (e.g., a pixel-wise conditional mean, not a literal average of every future) or does not apply to deterministic settings. MOSTLY FALSE: blurriness occurs only in narrow settings and is presented as general, or the evidence shows L2-trained predictors are often not blurry. FALSE: evidence shows least-squares training does not generally cause blurry predictions.

Set before the research began.

1. E1 Deep Multi-Scale Video Prediction Beyond Mean Square Error mlanthology.org (ICLR 2016 proceedings listing) · 2016 Own record

   “To deal with the inherently blurry predictions obtained from the standard Mean Squared Error (MSE) loss function, we propose three different and complementary feature learning strategies”

2. E2 Folded Recurrent Neural Networks for Future Video Prediction Oliu, Selva and Escalera (Universitat Oberta de Catalunya / Universitat de Barcelona), ECCV 2018 · 2018

   “Left unchecked, this results in blurry predictions averaging the space of possible futures.”

3. E3 Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning Moerland, Broekens and Jonker (TU Delft), arXiv · 2017

   “The point prediction of the function approximator will be the conditional expectation of the outcome distribution”

4. E4 Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning Moerland, Broekens and Jonker (TU Delft), arXiv · 2017

   “Oh et al. [25] shows MSE training does work well in high-dimensional, deterministic domains.”

Only E1, which is not independent of the subject, directly links MSE training to blurry video predictions. The independent items describe blurry predictions that average possible futures (E2) and point predictions equal to a conditional expectation (E3), but their quoted passages do not name least-squares training as the cause. E4 says only that MSE training works well in deterministic domains.