If the paddle in the game Breakout is moved by three pixels, a deep-learning system trained on the game fails.

Judge as of 2019-09-20. True: published research shows that deep RL agents trained on Breakout (e.g. DQN-style) fail badly when the paddle is shifted by a few pixels, e.g. Vicarious's 2017 Schema Networks paper showing failure when the paddle was moved up a few pixels. Mostly true: research shows such fragility under a small paddle shift, but the exact number of pixels or the perturbation differs, e.g. a vertical shift of a different small size. Mixed: the agents degraded under the shift, but only partly rather than 'falling apart'. Mostly false: the brittleness was shown for other perturbations, not for a paddle shift, or only for a few specific models. False: research shows trained agents handle such shifts robustly, or no such experiment exists and the evidence contradicts the claim.

Set before the research began.

1. E1 Schema Networks: Zero-shot Transfer with a Generative Causal Model of Intuitive Physics PMLR (Proceedings of Machine Learning Research), ICML 2017 · 17 July 2017

   “We compare Schema Networks with Asynchronous Advantage Actor-Critic and Progressive Networks on a suite of Breakout variations, reporting results on training efficiency and zero-shot generalization, consistently demonstrating faster, more robust learning and better transfer.”

2. E2 PaperWeekly/DL-Critical-Appraisal.md at master · Mycenae/PaperWeekly GitHub (Mycenae/PaperWeekly repository) Own record

   “a team of researchers at Vicarious showed that a more efficient successor technique, DeepMind’s Atari system [Asynchronous Advantage Actor-Critic; also known as A3C], failed on a variety of minor perturbations to Breakout (Kansky et al., 2017) from the training set, such as moving the Y coordinate (height) of the paddle, or inserting a wall midscreen.”

3. E3 Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation PMLR 97 (Proceedings of the 36th International Conference on Machine Learning) · 2019

   “However, when a network trained on the original game is presented with the game variants, it fails completely, reaching to a maximum score of only 3 points.”

4. E4 PaperWeekly/DL-Critical-Appraisal.md at master · Mycenae/PaperWeekly GitHub (mirror of Gary Marcus, 'Deep Learning: A Critical Appraisal') Own record

   “failed on a variety of minor perturbations to Breakout (Kansky et al., 2017) from the training set, such as moving the Y coordinate (height) of the paddle, or inserting a wall midscreen.”

5. E5 Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation Proceedings of Machine Learning Research (ICML 2019, PMLR 97) · 2019

   “However, when a network trained on the original game is presented with the game variants, it fails completely, reaching to a maximum score of only 3 points.”

6. E6 Zero-shot Transfer with a Generative Causal Model of Intuitive Physics Proceedings of Machine Learning Research (PMLR), vol. 70 (ICML 2017); authors affiliated with Vicarious AI · 2017

   “Offset Paddle (Fig. 1d): The paddle is shifted upward by a few pixels.”

7. E7 Zero-shot Transfer with a Generative Causal Model of Intuitive Physics Proceedings of Machine Learning Research (PMLR), vol. 70 (ICML 2017); authors affiliated with Vicarious AI · 2017

   “deep neural network-based models are easily fooled”

8. E8 Transferring Deep Reinforcement Learning with Adversarial Objective and Augmentation arXiv

   “We designed 4 different levels with increasing difficulties by shrinking the paddle width, the different paddle widths are: 30px width (level-1), 20px width (level-2), 10px width (level-3) and 5px width (level-4).”

The Kansky et al. (2017) paper defines an 'Offset Paddle' Breakout variation with the paddle shifted upward by a few pixels [E6], and compares Schema Networks with A3C on a suite of Breakout variations [E1]. It also states generally that deep networks are 'easily fooled' and that Schema Networks transfer more robustly [E1, E7]. None of the quoted independent text reports how A3C actually performed on the paddle-shift variation, so it is unclear whether performance collapsed or only partly degraded. The Gamrian et al. quotes [E3, E5] do not say which variants they used, the reports of a failure under the paddle shift come only from Marcus's own paper [E2, E4], and E8 concerns paddle width, not a paddle shift.