Human raters make systematic, regular, compactly describable and predictable errors.

Judge as of 2022-06-10. True: research (e.g. on RLHF and reward modelling, annotator bias, cognitive biases in human judgement) shows that human raters make errors that are systematic, i.e. predictable and describable patterns such as length bias, preference for confident or flattering answers, and known cognitive biases, rather than only random noise. Mostly true: systematic errors are documented, but 'compactly describable' or 'predictable' is somewhat overstated for some error types. Mixed: the evidence shows rater errors are about equally random noise and systematic bias. Mostly false: rater errors are mostly random, with only minor systematic components. False: the evidence shows rater errors are not systematic.

Set before the research began.

1. E1 Comparing verbosity bias in Human and LLM based evaluators Amanda Dsouza (personal blog, amy12xx.github.io) · 26 July 2023

   “Similarly, on the Vicuna benchmark, 68.82% of the time, annotators preferred outputs with longer generations, between competing models (controlled for position bias).”

2. E2 Comparing verbosity bias in Human and LLM based evaluators Amanda Dsouza (personal blog, amy12xx.github.io) · 26 July 2023

   “We compare the model generated outputs in MT-bench evaluated by human experts and GPT-4, and find that the rate of preferring longer generated text outputs is nearly the same by both, while being slightly higher for GPT-4 (Human eval - 61.22% versus GPT-4 eval - 65.95%).”

3. E3 A survey of reward hacking in agentic large language model systems - Discover Artificial Intelligence Springer (Discover Artificial Intelligence) · 17 August 2026

   “Human preference labels carry systematic biases (verbosity and agreeableness) that the reward model encodes as training signals, which are amplified by RL optimization.”

4. E4 Explaining Length Bias in LLM-Based Preference Evaluations arXiv (academic preprint) · 30 March 2023

   “However, attributing human preference solely to response length is an oversimplification, as length is merely a superficial factor in how humans judge the quality of a sentence.”

E1 and E2 report that human annotators chose the longer output in 68.82% (Vicuna) and 61.22% (MT-bench) of pairwise comparisons, a directional, measurable bias rather than random noise. E3, a survey, states that human preference labels carry systematic biases toward verbosity and agreeableness. E4 cautions that putting human preference down to length alone is an oversimplification, which suggests that length bias alone does not fully describe how raters judge. These sources postdate the claim, but they describe a property of human judgement that is not tied to any date, so on balance they support the claim with some qualification.