Almost all bias in AI systems comes from humans.

Judge against the AI research literature as of 2020-09-23. True: authoritative surveys or expert consensus say nearly all AI bias comes from humans, meaning human-generated or human-labelled data, human design choices and human framing of problems. Mostly true: the literature names human sources as the dominant cause but also gives some non-negligible non-human sources (e.g., algorithmic or statistical artifacts such as sampling, optimisation or aggregation effects), while still treating these as largely the result of human decisions. Mixed: the literature treats human and non-human or technical sources as comparably important. Mostly false: human sources are a major cause, but the literature describes substantial bias arising from technical or algorithmic mechanisms apart from human input, so 'almost all' overstates it. False: the evidence shows most AI bias comes from sources other than humans.

Set before the research began.

1. E1 A Framework for Understanding Unintended Consequences of Machine Learning (arXiv:1901.10002v1) arXiv preprint by Harini Suresh and John V. Guttag (MIT); hosted on a Duke course site · 28 January 2019

   “And it is not just the data that causes problems. The ML pipeline involves a series of choices and practices, from evaluation methodology to model definition, that can lead to unwanted effects.”

2. E2 A Framework for Understanding Unintended Consequences of Machine Learning (arXiv:1901.10002v1) arXiv preprint by Harini Suresh and John V. Guttag (MIT); hosted on a Duke course site · 28 January 2019

   “In data that is affected by human decision-makers, we often see human decisions used as proxies, introducing measurement bias.”

3. E3 A Framework for Understanding Unintended Consequences of Machine Learning (arXiv:1901.10002v1) arXiv preprint by Harini Suresh and John V. Guttag (MIT); hosted on a Duke course site · 28 January 2019

   “Issues that arise in image recognition, for example, are often related to selection or evaluation bias. Images themselves tend to be objective, but they may not equally represent the entire space of images that we care about.”

4. E4 A Framework for Understanding Unintended Consequences of Machine Learning (arXiv:1901.10002v1) arXiv preprint by Harini Suresh and John V. Guttag (MIT); hosted on a Duke course site · 28 January 2019

   “Historical bias arises even if the data is perfectly measured and sampled, if the world as it is leads a model to produce outcomes that are not wanted.”

A single framework paper names human sources of bias, such as human decisions used as proxies [E2]. It also describes other sources: choices across the ML pipeline [E1], selection and evaluation bias [E3], and historical bias that arises from the world as it is even with perfect data [E4]. It does not say whether these other sources are largely the result of human decisions or how much each contributes, so the evidence does not decide between the 'mostly true', 'mixed' and 'mostly false' bands.