The amount of image training data available for self-supervised learning is essentially unlimited, with billions of new images per day.

Judge as of 2021. True: credible sources from around 2021 show that the amount of image data available for self-supervised learning (images that need no labels) was so large that it was not a practical limit, with billions of new images shared online per day (e.g. platform upload statistics on the order of billions per day). Mostly true: the volume of images was vast and effectively not limiting, but the 'billions per day' figure is somewhat overstated (e.g. hundreds of millions to about one billion per day). Mixed: images were plentiful, but credible evidence from that period shows that usable data (after licensing, privacy, curation or quality filtering) was a real constraint, so 'essentially unlimited' is only partly right. Mostly false: daily image volumes were much lower than claimed, or usable data was clearly a binding limit, and the claim turns a large figure into 'unlimited'. False: evidence shows image data for self-supervised learning was scarce or tightly limited in 2021.

Set before the research began.

1. E1 Now Patched Vulnerability in WhatsApp could have led to data exposure of users - Check Point Research Check Point Research · 2 September 2021

   “Approximately 55 billion messages are sent daily over WhatsApp, with 4.5 billion photos and 1 billion videos shared per day.”

2. E2 90% of young women report using a filter or editing their photos before posting ScienceDaily (City University London press release) · 8 March 2021

   “"With nearly 100 million photos posted every single day on Instagram alone, we have never been such a visually dominated society.”

3. E3 Self-supervised Pretraining of Visual Features in the Wild arXiv (Goyal, Caron, Lefaudeux, et al.; Facebook AI Research) Own record

   “Further, these methods are amenable to online learning, making them perfect candidates for training large models on unlimited data.”

4. E4 Self-supervised Pretraining of Visual Features in the Wild arXiv (Goyal, Caron, Lefaudeux, et al.; Facebook AI Research) Own record

   “For our billion scale pretraining, we consider a dataloader that directly samples random, public, and non-EU images from Instagram.”

E1, an independent 2021 source, reports about 4.5 billion photos shared per day on WhatsApp alone, which supports the 'billions per day' scale. E2 reports nearly 100 million photos posted daily on Instagram. However, E4 shows that Facebook AI Research's billion-scale self-supervised pretraining sampled random, public, non-EU Instagram images. So at least one billion-scale pretraining effort sampled only a subset of uploaded images [E4]. Still, nothing in the evidence shows that usable data was a binding limit: E4 describes billion-scale pretraining and E3 calls these methods suited to training large models on unlimited data, which supports 'essentially unlimited' in practice but with qualifications.