Scott Alexander said that, if he understood it right, in the study the average forecaster did worse than the average expert, but Tetlock used clever aggregation methods for his superforecasters while the CIA did not use such methods for its experts.

The proposition is about what Scott Alexander said. True: his post of about 2022-03-14 contains this statement, hedged with 'if I'm understanding this right': the average forecaster did worse than the average expert, Tetlock used clever aggregation for his superforecasters, and the CIA did not use such methods for its experts. Mostly true: the post makes this point with a minor difference in wording or detail. False: he did not make this statement, or said something substantially different. Whether his reading of the study is correct is not tested.

Set before the research began.

1. E1 Can Policymakers Trust Forecasters? | IFP Institute for Progress (IFP) · 7 March 2023

   “the aggregation method used for the forecasters is known to produce better results than the one used for the expert analysts.”

2. E2 Can Policymakers Trust Forecasters? | IFP Institute for Progress (IFP) · 7 March 2023

   “In particular, the amateur forecasts were weighted according to each forecaster’s past performance, while the expert market used weights proportional to each analyst’s current subjective confidence.”

The proposition concerns what Scott Alexander wrote in his post of about 2022-03-14, but none of the evidence quotes or cites that post. E1 and E2 are from a 2023 IFP article. They describe how the forecasters' and the expert analysts' predictions were aggregated in the study, but they do not record Scott Alexander's statement. Under the criteria, the accuracy of his reading of the study is not tested, so this evidence cannot settle whether he made the statement.