Match the target and forecast time
Compare predictions of the same difficulty change at the same target block, issued at comparable times. A later forecast can use information that an earlier one did not have.
Keep each forecast as originally issued, with its stated range, input block height, source, model version and outcome definition. Record its publication time, including an independently verifiable capture when timing matters. Group forecasts by lead time. Record failed and missing forecasts, as well as the exclusion rules; selecting only successful predictions would change the comparison.
Point errors and ranges answer different questions
Absolute error is the size of the gap between a forecast and its outcome. Mean absolute error averages those gaps across the forecasts tested. A smaller mean means the point estimates were closer on average in that sample; it does not establish performance in another period.
Ranges answer a different question: how often did they contain the outcome? Read that share alongside their width, since a wide range can contain many outcomes while telling you little. Testing a stated confidence level needs enough independent observations over a defined period.
What the worked example can show
The linked example shows each step of the error calculation. Its forecasts and outcomes were made up for the exercise. They are neither captured predictions nor network observations, and they do not represent Doefin or another estimator.
No verified comparison of public forecasters is available here. The example establishes no provider ranking, accuracy advantage, volatility estimate or record of which forecasters publish ranges.
