Setting aside the 50% bin, which he called meaningless, Scott Alexander said his main calibration deviation in 2016 was some underconfidence at the 70% level.

Judge as of 2016-12-31, against his published 2016 calibration results. True: his calibration table shows the 70% bin (excluding 50%) had the largest gap from perfect calibration, with the share of correct predictions above 70% (underconfidence). Mostly true: the 70% bin shows underconfidence and is the largest gap or roughly tied for it. Mixed: the 70% bin is underconfident but another bin shows a clearly larger deviation. Mostly false: the 70% bin is close to calibrated or only marginally off while other bins deviate more. False: the 70% bin was overconfident (correct share below 70%).

Set before the research began.

1. E1 Scott Alexander's Self-Evaluations - Econlib Econlib (reader comment by 'richard', comment dated Apr 6 2017 at 3:09pm) · 6 April 2017

   “Of 70% predictions, I got 13 right and 3 wrong, for a score of 81%”

2. E2 Scott Alexander's Self-Evaluations - Econlib Econlib (reader comment by 'richard', comment dated Apr 6 2017 at 3:09pm) · 6 April 2017

   “Of 50% predictions, I got 8 right and 5 wrong, for a score of 62%”

3. E3 Scott Alexander's Self-Evaluations - Econlib Econlib (reader comment by 'richard', comment dated Apr 6 2017 at 3:09pm) · 6 April 2017

   “Of 60% predictions, I got 12 right and 9 wrong, for a score of 57%”

4. E4 Evaluating Predictions in Hindsight Zvi Mowshowitz (thezvi.wordpress.com) · 16 April 2020

   “If they happen more often than that, you’re under-confident. If they happen less than that, you’re over-confident.”

5. E5 2017 Predictions: Calibration Results Slate Star Codex · 2 January 2018 Own record

   “Last year my main concern was that I was underconfident at 70%.”

6. E6 Review of "Rationality: AI to Zombies": Rationality vs. Antifragility wearenotsaved.com · 25 November 2017

   “Alexander got about 90% of his 90% predictions correct, so the system works, and everything is fine right?”

A reader comment on Econlib copies figures showing 70% predictions at 13 right and 3 wrong (81%) and 60% predictions at 57% (E1, E3). That would mean underconfidence at the 70% level. However, neither the quote nor the source title says which year's results these are, so they cannot be tied to 2016. No evidence gives the 80%, 95% or 99% results, and E6 only says roughly 90% of 90% predictions came true, so it is not possible to show that the 70% gap was the largest. E5 is Scott Alexander restating his own earlier conclusion, so it does not count as evidence that the claim is true.