Accurate Forecast Testing
I found a site that explains what exactly I need to do for my data, but it is not in R. Can anyone suggest how I could create this in R?
http://people.duke.edu/~rnau/three.htm
I need to search for MSE, MAE, MAPE, ME, MPE, SSE to check the accuracy of the predictions and this page is the closest I found to explain how to do it.
data<-c(79160.56266,91759.73029,91186.47551,106353.8192,70346.46525,80279.15139,82611.60076,131392.7209,93798.99391,105944.7752,103913.1296,154530.6937,110157.4025,117416.0942,127423.4206,156751.9979,120097.8068,121307.7534,115021.1187,150657.8258,113711.5282,115353.1395,112701.9846,154319.1785,116803.545,118352.535)
forecasts<-c(118082.3,157303.8,117938.7,122329.8) # found using arima
(if you check this question can you explain why exactly)
source to share
Here are some examples to help you get started using a dataset UKNonDurables
from a package AER
. This package accompanies the book Applied Econometrics with R , which is a pretty good introductory book of econometrics, especially for people with little programming experience.
library(forecast)
library(AER)
##
data("UKNonDurables")
## alias for convenience
Data <- UKNonDurables
## split data into testing and training
train <- window(
Data,
end=c(1975,4))
test <- window(
Data,
start=c(1976,1))
## fit a model on training data
aaFit <- auto.arima(
train)
## forcast training model over
## the testing period
aaPred <- forecast(
aaFit,
h=length(test))
##
> plot(aaPred)
## extract point forecasts
yHat <- aaPred$mean
## a few functions:
## mean squared (prediction) error
MSE <- function(y,yhat)
{
mean((y-yhat)**2)
}
## mean absolute (prediction) error
MAE <- function(y,yhat)
{
mean(abs(y-yhat))
}
## mean absolute percentage (prediction) error
MAPE <- function(y,yhat,percent=TRUE)
{
if(percent){
100*mean(abs( (y-yhat)/y ))
} else {
mean(abs( (y-yhat)/y ))
}
}
##
> MSE(test,yHat)
[1] 9646434
> MAE(test,yHat)
[1] 1948.803
> MAPE(test,yHat)
[1] 3.769978
So as I said, some or all of the above functions probably exist in the R base or inside external packages, but they tend to be simple formulas that are trivial to implement. Try to deal with this and / or adapt them to better suit your needs.
Edit:
As Mr. Hyndman noted below, his package forecast
includes a feature accuracy
that provides a very convenient way to sum the GOF measures of time series models. Using the same data from above, you can easily assess the suitability of an object forecast
over the training and testing period:
> round(accuracy(aaPred,Data),3)
ME RMSE MAE MPE MAPE MASE ACF1 Theil U
Training set 2.961 372.104 277.728 0.001 0.809 0.337 0.053 NA
Test set 1761.016 3105.871 1948.803 3.312 3.770 2.364 0.849 1.004
(where round(...,3)
only used to make the result fit well in this post). Or, if you only want to study these measures for the forecast period, you can call something like this:
> accuracy(yHat,test)
ME RMSE MAE MPE MAPE ACF1 Theil U
Test set 1761.016 3105.871 1948.803 3.312358 3.769978 0.8485389 1.004442
source to share