Accurate Forecast Testing

I found a site that explains what exactly I need to do for my data, but it is not in R. Can anyone suggest how I could create this in R?

I need to search for MSE, MAE, MAPE, ME, MPE, SSE to check the accuracy of the predictions and this page is the closest I found to explain how to do it.

forecasts<-c(118082.3,157303.8,117938.7,122329.8) # found using arima


(if you check this question can you explain why exactly)


source to share

1 answer

Here are some examples to help you get started using a dataset UKNonDurables

from a package AER

. This package accompanies the book Applied Econometrics with R , which is a pretty good introductory book of econometrics, especially for people with little programming experience.

## alias for convenience
Data <- UKNonDurables
## split data into testing and training
train <- window(
test <- window(
## fit a model on training data
aaFit <- auto.arima(
## forcast training model over
## the testing period
aaPred <- forecast(
> plot(aaPred)


enter image description here

## extract point forecasts
yHat <- aaPred$mean
## a few functions:
## mean squared (prediction) error
MSE <- function(y,yhat)
## mean absolute (prediction) error
MAE <- function(y,yhat)
## mean absolute percentage (prediction) error
MAPE <- function(y,yhat,percent=TRUE)
    100*mean(abs( (y-yhat)/y ))
  } else {
    mean(abs( (y-yhat)/y ))
> MSE(test,yHat)
[1] 9646434
> MAE(test,yHat)
[1] 1948.803
> MAPE(test,yHat)
[1] 3.769978


So as I said, some or all of the above functions probably exist in the R base or inside external packages, but they tend to be simple formulas that are trivial to implement. Try to deal with this and / or adapt them to better suit your needs.

Edit: As Mr. Hyndman noted below, his package forecast

includes a feature accuracy

that provides a very convenient way to sum the GOF measures of time series models. Using the same data from above, you can easily assess the suitability of an object forecast

over the training and testing period:

> round(accuracy(aaPred,Data),3)
                   ME     RMSE      MAE   MPE  MAPE  MASE  ACF1 Theil U
Training set    2.961  372.104  277.728 0.001 0.809 0.337 0.053        NA
Test set     1761.016 3105.871 1948.803 3.312 3.770 2.364 0.849     1.004


(where round(...,3)

only used to make the result fit well in this post). Or, if you only want to study these measures for the forecast period, you can call something like this:

> accuracy(yHat,test)
               ME     RMSE      MAE      MPE     MAPE      ACF1 Theil U
Test set 1761.016 3105.871 1948.803 3.312358 3.769978 0.8485389  1.004442




All Articles