The Kaggle Workbook by Konrad Banachewicz

Author:Konrad Banachewicz [Banachewicz, Konrad]
Language: eng
Format: epub
Tags: data analysis; artificial intelligence; data analytics; ai/ml books; ai/ml; kaggle machine learning; python data analysis
Publisher: Packt Publishing
Published: 2023-01-13T00:00:00+00:00

Understanding the Evaluation Metric

The accuracy competition introduced a new evaluation metric: Weighted Root Mean Squared Scaled Error (WRMSSE). The metric evaluates the deviation of the of the point forecasts around the mean of the realized values of the series being predicted:

where:

n is the length of the training sample

h is the forecasting horizon (in our case it is h=28)

Yt is the sales value at time t, is the predicted value at time t In the competition guidelines (https://mofc.unic.ac.cy/m5-competition/), in regard of WRMSSE, it is remarked that:

The denominator of RMSSE is computed only for the time-periods for which the examined product(s) are actively sold, i.e., the periods following the first non-zero demand observed for the series under evaluation

The measure is scale independent, meaning that it can be effectively used to compare forecasts across series with different scales.

In contrast to other measures, it can be safely computed as it does not rely on divisions with values that could be equal or close to zero (e.g. as done in percentage errors when Yt = 0 or relative errors when the error of the benchmark used for scaling is zero).

The measure penalizes positive and negative forecast errors, as well as large and small forecasts, equally, thus being symmetric.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.