Evaluate demand estimates against actual sales at the product and location level. Use a frozen forecast, a simple benchmark and reconciled error measures before accepting an explanation.
Artificial intelligence demand forecasting is useful only when the underlying estimates survive a comparison with actual demand. Save the forecast before the selling period, calculate errors on matching product and location rows, and compare the result with a simple benchmark. The external AI model can explain those calculations; persuasive wording does not establish accuracy.
Bring the forecast export and the actual-sales export used by the demand planning team. Each needs a product ID, location ID, selling period and quantity in a consistent unit. The forecast also needs a version date, forecast horizon and method label. Actuals need a completeness check and a clear rule for returns.
Record how the estimates were produced. A driver method might use confirmed orders plus an approved estimate of additional demand. Statistical models fitted on your data might use historical observations and calendar effects. Either way, retain the assumptions and the estimates that existed when the planning decision was made.
For the wider method choices, start with demand forecasting. This review is narrower: it tests an existing set of estimates rather than building another forecasting method.
Use a benchmark that was also available at the forecast date, such as the previous comparable week's units. A historical average can be another benchmark, but it is not automatically suitable: NIST's discussion of forecasting with averages explains why trends make an overall average misleading.
Keep one comparison row per product, location, period and forecast version. Aggregate invoice lines to that grain before joining them to the forecast. Otherwise, one estimate can be repeated against several invoice lines and counted several times.
Keep unmatched records visible. An actual sale without a forecast is different from a forecast for a product that recorded no sales. A missing actual-sales extract is different from confirmed zero demand. Do not replace all three with zero.
Hold the evaluation window and the horizon constant. Comparing a forecast made immediately before shipment with one made before a supplier commitment gives the later forecast an information advantage. The demand forecasting process should define who freezes each version and when actuals are considered complete.
A US distributor reviews a completed selling week. Quantities are units; there are no returns, stockouts or missing records in this example. The candidate estimates came from the team's documented method. The benchmark is the previously available comparable week's sales.
Use forecast minus actual as the error sign, so positive means overforecasting. This convention must remain consistent throughout the report.
| Product | Location | Actual units | Candidate units | Benchmark units | Candidate error | Candidate absolute error | Benchmark absolute error |
|---|---|---|---|---|---|---|---|
| A | Dallas | 100 | 110 | 120 | 10 | 10 | 20 |
| A | Phoenix | 80 | 70 | 95 | -10 | 10 | 15 |
| B | Dallas | 40 | 55 | 60 | 15 | 15 | 20 |
| B | Phoenix | 60 | 45 | 70 | -15 | 15 | 10 |
| Total | 280 | 280 | 345 | 0 | 50 | 65 |
With these columns in A through H and detail rows 2 through 5, enter these formulas in F2, G2 and H2, respectively:
=D2-C2
=ABS(F2)
=ABS(E2-C2)
Fill down. Then calculate portfolio measures from the detail totals:
Mean absolute error, units = Sum of absolute errors / Number of comparison rows
WAPE = Sum of absolute errors / Sum of actual units
Signed bias percentage = Sum of forecast-minus-actual errors / Sum of actual units
These WAPE formulas assume nonnegative actual demand. Returns should follow the agreed demand definition rather than being mixed into the denominator without review.
Use the following formulas for candidate MAE, candidate WAPE, candidate bias and benchmark WAPE, respectively:
=AVERAGE(G2:G5)
=SUM(G2:G5)/SUM(C2:C5)
=SUM(F2:F5)/SUM(C2:C5)
=SUM(H2:H5)/SUM(C2:C5)
| Measure | Candidate | Benchmark |
|---|---|---|
| Absolute error, units | 50 | 65 |
| Mean absolute error, units | 12.50 | 16.25 |
| WAPE | 17.9% | 23.2% |
| Signed bias | 0.0% | 23.2% |
Percentages are rounded to one decimal place from unrounded calculations. The candidate improves WAPE by 5.4 percentage points, calculated as (65-50)/280. That is evidence about this completed comparison, not proof that the method will outperform across seasons or locations.
First, confirm that the actual total of 280 units agrees with the sales export after the documented exclusions. Confirm that candidate units total 280 and benchmark units total 345 in their frozen versions. Finally, check that summed signed error equals forecast total minus actual total: 280-280=0.
Zero bias does not mean a perfect forecast. The candidate still misses by 50 units in absolute terms. Overestimates at some product-location combinations cancel underestimates elsewhere. The existing SKU and site forecast-bias guide covers the detailed diagnosis; use it when the aggregate looks acceptable but particular locations keep missing.
WAPE also needs boundaries. Rob Hyndman's explanation of WAPE notes its undefined denominator when all actuals are zero and limitations when comparing trending or seasonal series. Use it here as a descriptive measure on one fixed comparison set. Do not assume improving WAPE across different periods proves an improved forecasting method.
For sparse product histories, follow the separate intermittent demand forecasting guidance. Keep unit errors visible when percentage measures cannot be calculated.
The planning owner decides which method to retain, using repeated comparable reviews and the cost of stockouts or excess inventory. For the broader distinction between calculations and interpretation, see AI analytics.
Bring a sample of your own frozen forecast and actual-sales exports; Covirage maps the columns, and its tools compute and check the comparison figures. The external AI model explains the results with every figure traced to its rows. Create a free Covirage account and set up your business before bringing your files.
See forecast analysis for the supporting approach to reviewing forecast performance.
Compare a forecast saved before the selling period with actual results for matching products, locations and periods. Calculate absolute error and signed bias, then compare both with a simple benchmark using the same rows and forecast horizon.
Yes. Positive and negative errors can cancel when summed, leaving zero net bias even when individual estimates miss substantially. Absolute error prevents that cancellation, so review error size and direction together before making inventory decisions.
Do not silently exclude them. A zero can represent genuine inactivity, a stockout or a missing record. Investigate its meaning first. Individual percentage errors are undefined at zero actual sales, so retain absolute-unit errors and label any undefined percentage clearly.