Dimensiones del área de estudio

4.7.1. Overview:

Various models and methods of forecasting may result in varied forecasted values and each methodology requires different means of assessment. (G. C. Wang and Jain, 2003) Wang, et al, (2003), recognizes some of these forecasting models which are practiced customarily: Logistic Regression: Linear Regression, Auto regression and Auto regression Moving Average (Nassirtoussi et al.). The groundwork of this thesis is based on Auto Regression Integrated Moving Average (ARIMA) model. However, the error metrics help us determine the precariousness of a forecast model or method. Due to this, it is noteworthy that the efficacy of any prediction system relies heavily on choosing proper metrics. Likewise, the traits of the object of concern along with the characteristics of performance metrics should be apprehended before selecting a suitable model for the application of metrics to particular circumstances. Moreover, Complex metrics are not the first priority of any practitioner, since these procedures can complicate a forecast. For example, financial forecasting is complicated and decisions depend upon motivation and explanations. Thus a good error metric would be easy to decrypt.

The caliber of a prediction method can be assumed by applying numerous metrics. Hyndman and Koehler (2006) noted that absolute errors or squared errors are more popular for scale- dependent calculations. These include Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Median Absolute Error (MAE). However, The purpose of this thesis is to deduce metrics in accordance to prediction method to tackle the time series. This can be achieved by understanding the stock market with regards to measurements. Hence, when considering the stock market, the relative efficient market is concerned with two things: firstly, to govern the difference between the forecasted values and actual values, secondly, to check the strength of the linear relationship between dependent and independent values.

Resolution of a suitable metric which abides by the above prerequisites of the stock market is possible if the primary aspects of good metric are realized. The equations which present the difference between the actual and predicted value are called as Error metrics. The value of outcome from comparing the forecasted values to the actual values is the deciding factor a model’s performance. If the outcome is minimal, this would mean that the forecast model

is dependable for the given circumstances and is an efficient instrument for predicting future movements. Thus, Mean Absolute Error (MAE), Mean Square Error (MSE), Root Mean Square Error (RMSE) and Mean Absolute Percent Error (MAPE) are used to evaluate the performance of these prediction models. These are efficient means of prognosis for movement and the assessment of share price and indices. Formula of these evaluation measures are shown in below:

Mean Absolute Error (MAE): MAE=_𝐍𝟏∑ 1𝐏𝐭L𝐏›𝒕1

𝐏𝐭

𝐍

𝐈@𝟏 (4.7.1)

Mean Square Error (MSE): MSE= _𝑵𝟏∑𝑵 (𝑷_𝒕− 𝑷›_𝒕)𝟐

𝑰@𝟏 (4.7.2)

Root Mean Square Error (RMSE):

RMSE = ž_𝐍𝟏∑𝐍 (𝐏_𝐭− 𝐏›_𝐭)𝟐

𝐈@𝟏 (4.7.3)

Mean Absolute Percent Error (MAPE):

MAPE =𝟏𝟎𝟎_𝐍 ∑ 1𝐏𝐈L𝐏›𝐈1

𝐏𝐈

𝐍

𝐈@𝟏 (4.7.4)

Where N is the number of forecasting periods, Pt is the actual stock price at period t, and Pt ̂is the forecasting stock price at period t.

4.7.2.Performance Evaluation Metric:

This research involves the evaluation of performance of the prediction model, this would be achieved by studying three indices from GCC market involving five individual companies from Saudi stock market. There are four common errors involved in achieving this, which are the Mean Absolute Error (MSE), Mean Square Error (MSE), Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE). Varied time series and various samples are applied

across varied error metric methods, and the results are compared so as to assume their accuracy.

“Each of these measures have their relative advantages and disadvantages. The MAE has no scale by which they can be relatively measured. The MAPE is somewhat scale dependent that when forecasting very low values or integers such as a none or two- the size of the measure is easily inflated to 100% or more. Therefore, when using the MAPE, it is important to accompany it with the MAE to provide a sense of balance”(Yaffee, 2010) in addition to the issue with RMSE, when large positive errors are divided over larger negative errors, the result may be approaching 0; MAE is used in place of RMSE in such a case. RMSE may be used when large errors are not valuable. RMSE imparts high weightage to large errors through the squaring process. It responds drastically to the infrequent large errors when squared (Decision 411, 2010). All the differences are weighed equally in MAE. On the other hand, RMSE will always be larger than or equal to MAE. Therefore, for a given set of forecasts, the two can be utilized together to derive the variation in the errors. The RMSE and MSE are more sensitive to outliers than MAE. However, they have been common, because of their theoretical relevance in econometrics modelling. (Jon Scott Armstrong, 2001).

MSE results in squared errors and the resultant will always be a squared value, thus RMSE is preferred over it. RMSE provides a more accurate forecast as square rooting the resultant reverses the effect of squaring the errors and the result is not exaggerated. On the other hand, (Willmott and Matsuura, 2005) gave preference to MAE over RMSE terming it a better metric for the purpose of Average Error and argued that RMSE could result in a deceptive result. However, model evaluation studies frequently utilize both methods. The arguments put forth by Willmott and Matsuura (2005) and Willmott et al., (2009) cannot be disagreed with, but RMSE cannot be ignored completely. Even though the researchers in the above mentioned study employed MAE over RMSE to provide a representation of their model evaluation statistics, MAE presents an underrated resultant effect making it least favourable in the various aspects like decision making. For that purpose, (J Scott Armstrong and Collopy, 1992) Scott and Armstrong and Collopy (1979) proposed using Mean Absolute Percentage Error (MAPE) which is capable of giving a close percentile representation for decision making. It is dependable and outliers are prevented in applying this error.

MAPE was employed as a primary measure in the M-competition (Makridakis et al., 1982). Hanke (1995) and (Bowerman et al., 2004) are some of the numerous literatures that support

the use of MAPE. Percentage Errors are used in differentiating between a varied set of forecast performance data. It is one of the most widely practiced evaluations metric for forecasting due to its scale independent nature (Klimberg et al., 2010). On the downside, MAPE has been criticized for not being the best means for decision making; however, they provide an accurate magnitude of the movement. Makridakis has been a noted critic of the use of Percentage Errors. Makridakis et al., (1998), along with Wheelwright and Hyndman (1998), avoided using MAPE in particular situations. It was observed that MAPE tends to apply a heavier penalty on positive errors than on negative errors. This lead to the use of “symmetric” measures(Makridakis, 1993). Ord and Fildes (2013) recommended the use of MAPE in exclusive circumstances when the values are bigger than zero. This is the most prominent drawback of applying this error. Furthermore, Makridakis in 1998 noted that Percentage Errors are capable of drawing a meaningful zero. This draws a hindrance in evaluating percentage errors when time series includes zero (Fildes et al., 1998). This applies to situations with time series values which are negligible or approaching zero. In such a case, assuming the percentage error serves no purpose.