Scores by kind of claim
Each kind of claim is scored on its own, out of 100: the share that held up, with the range it likely falls in. Fewer resolved claims give a wider range.
-
Factual claims
Statements about the present or the past, checked against the record: did they hold up?
83 77 88 -
Predictions
Statements about events outside the person's control, checked once they fell due: did they come true?
69 49 87 -
Promises
Commitments to the person's or organisation's own future action, with something to deliver: was it delivered?
69 49 87
All types combined: 81 (likely 76 to 86, high confidence), from 87 claims: 77 factual claims, 5 predictions, 5 promises. It mixes kinds of claim in different proportions for different people, so compare people kind by kind.
Sam Altman's claims
41 claims
“Most machine intelligence development involves a “fitness function”—something the program tries to optimize.”
Sam Altman, blog author February 2015 Machine intelligence, part 1 (Sam Altman's blog)
Most machine intelligence development involves a fitness function that the program tries to optimize.
Resolved true The record shows it happened or was so.
The second judge would not resolve this claim; the verdict shown is the first judge's.
What would count as true or false
Judge as of February 2015. True: standard machine learning practice at that time mostly involved optimising an objective (a loss, reward or fitness function); counting loss and objective functions as 'fitness functions' in the general sense the claim intends. Mostly true: this holds for the great majority of methods, and 'fitness function' is a loose but fair term for the objective. Mixed: a large share of significant machine intelligence approaches at the time did not optimise an explicit objective. Mostly false: optimisation-based methods were a minority of the field. False: the field generally did not involve optimising any objective function.
Evidence
- Deep learning Nature (review by Yann LeCun, Yoshua Bengio & Geoffrey Hinton) · 28 May 2015
“We compute an objective function that measures the error (or distance) between the output scores and the desired pattern of scores. The machine then modifies its internal adjustable parameters to reduce this error.”
- RLtut.key NIPS 2015 tutorial slides (Sutton and colleagues; lecture slides credit David Silver 2015) · 2015
“Seeking to maximize its cumulative reward in the long run”
- pypop/docs/sphinx/source/user-guide.rst at main · Evolutionary-Intelligence/pypop GitHub (Evolutionary-Intelligence/pypop open-source library documentation) · 26 January 2021
“First, an *objective function* (also called *fitness function* in evolutionary computation or *loss function* in machine learning) needs to be defined in the function form.”
Reasoning
The 2015 Nature review of deep learning (E1) describes the standard method: a system computes an objective function measuring error and adjusts its parameters to reduce it. The 2015 NIPS reinforcement learning tutorial (E2) describes agents maximising cumulative reward. E3 says objective, fitness and loss functions are names for the same thing across fields, which supports the general sense the criteria accept. Taken together, these independent sources show that optimising an objective was standard practice in the main machine learning approaches around the time of the claim.
Cite this claim
Sam Altman, “Most machine intelligence development involves a fitness function that the program tries to optimize.” (said February 2015). Verdict: resolved true. Reliability API, claim c_a022a45cf9ab, method 0.1.0+c876030c. https://epistacklabs.com/claims/c_a022a45cf9ab
Claims without a verdict
Is a verdict here wrong, or do you speak for Sam Altman? Reply on the claim's own page, with the form at its foot (how replies work). Verdicts are made automatically; how verdicts and scores are made.