How the reliability score works

The reliability score estimates how often what a company or a public figure says in public holds up against the public record: its statements of fact, its promises, the guidance it gives investors and its forecasts. It runs from 0 to 100. Every score comes with the claims behind it, and every claim with its exact quote, its source, the verdict and the evidence.

What the score measures

A score of 70 means: of the claims we checked, about 70% held up, after allowing for how few were checked. There is an overall score, which is the headline, and one for each kind of claim:

The overall score combines the four in proportion to how many claims each rests on.

How a report is made

1. Documents are chosen by rule, not by judgement. We list documents in which the subject itself speaks (annual reports, results announcements, interviews, its own publications, news reports quoting it), spread across the years, with some places kept for forward-looking documents such as plans and outlooks. Which documents are read is decided by a fixed, seeded rule, so nobody picks the ones likely to flatter or embarrass.
2. Every quote is checked word for word. Statements are copied from the documents, and code checks that each quote appears exactly in the page as published. A quote that fails is marked and never shown as verified.
3. Claims are sorted. Each checkable claim is marked as resolvable now, not yet due (its deadline is in the future), never resolvable (too vague) or checkable only against the subject's own internal records. Only resolvable claims are researched: up to 60 claims in all, taken in turn from each kind of claim (at most 60 of one kind), chosen by a seeded rule that spreads them across documents. Everything else is still counted and shown.
4. The standard is fixed before looking. For each claim, what would make it true, mostly true, mixed, mostly false or false is written down, with the date to judge it at, before any evidence is gathered, and is never rewritten afterwards. A claim is judged as stated: every qualifier and the deadline count, and an outcome after the deadline is not delivered on time.
5. Evidence is gathered from the public record. Searches look for evidence for and against equally, prefer sources independent of the subject, and respect each site's robots.txt. The claim's own source never counts as evidence for it. For a company's own figures (results, guidance outcomes), its audited or published accounts, filings and dated official announcements count; its ordinary statements do not. Every evidence quote is checked word for word against the page.
6. A model judges, a second checks. A judge model (Claude Opus) reads the claim, the fixed standard and only the verified evidence, and gives the verdict. A second judge from another model family (OpenAI GPT) does the same without seeing the first; where they differ, the claim is flagged and both verdicts are shown. Not finding a record is not taken as proof that something did not happen. When the evidence does not settle the claim, the verdict is "unresolvable", which is never counted against the subject.

How the score is calculated

Each claim with a verdict gets a value:

The score starts from a neutral prior and moves towards the evidence as checked claims accumulate:

score = 100 × (2 + sum of values) ÷ (4 + number of claims)

The prior is a mean of 50 worth 4 claims, so a thin record gives a middling score with a wide range, never an extreme one. The range shown with each score is its 80% interval: the middle 80% of the plausible values given the claims checked. Some worked examples:

Confidence follows a fixed rule on the number of checked claims and the width of the interval:

What the score does not tell you

• It describes what was checked, not everything the subject has ever said. The claims come from the documents the rule chose; the report says which, and how they were chosen.
• Every claim counts the same. A minor claim weighs as much as a central one, and an easy forecast as much as a bold one.
• Beating guidance counts as not met. "Revenue of EUR 15 to 16 billion" did not come true if revenue was 17 billion. The evidence beside each verdict shows what was reported, so a reader can see whether guidance was beaten or missed.
• Some verdicts rest only on the subject's own official records. For a company's own results that is how such facts are normally established, and the report marks those verdicts. A company's ordinary statements alone never settle a claim.
• "Unresolvable" is common. Some claims cannot be settled from public records. They are shown and counted, but not scored.
• Two model judges agreeing is consistency, not accuracy. How the method does against known outcomes is measured below.

How well it works

Measured 2026-10-14 on method version 0.1.0+6d6bb9f2. These are measures of agreement and consistency, not of accuracy in general: the sets are small and specific, and two model judges agreeing would only be consistency.

Agreement with known outcomes

A set of 104 claims from 13 listed companies whose outcome is a matter of public record: company guidance (for example "revenue of EUR 15 to 16 billion") set against the figure the company later reported, and dated company promises (a plant opened, a sale completed) with a documented outcome. The answers were recorded and checked in code before the method was run on the claims, and the method saw only each claim, as it would in a report.

"Right direction" counts a verdict of true or mostly true where the record shows the guidance met or the promise kept, and false or mostly false where it does not. Guidance that was beaten counts as not met, because the claim as stated did not come true.

Verdicts that hold on a re-run

The same companies run twice, the second time as a refresh would run: 152 of 164 verdicts (93%) were the same both times. Re-researching from scratch on the known-answer claims, 102 of 109 were the same.

How much each score rests on

Mechanical checks

Every evidence quote is checked word for word against the page it came from: 96% passed and only those reach the judges. Of the verdicts, 164 of 286 cite only figures and years that appear in the evidence they rest on or in the claim itself. This check is strict: a year mentioned in passing, or a figure worked out from the evidence (such as a percentage change), counts as not found.

Who is covered

Public organisations and public figures, from their public professional output only. Nothing about anyone's private life, family, health, home or location is collected. Serving politicians and officials in post are not covered, and party-political claims are shown but never scored. Only pages readable without logging in are used. A subject can reply to any claim; the reply is shown beside it.

Method version 0.1.0+5fd80fae. Any change to the method's instructions, rules or code changes this version, and every report says which version made it.