3. PRIMERA GENERACIÓN JAPONESAFig
3.3. DISCÍPULOS DE FRANK LLOYD WRIGHT
MAPE (refer to Figure 12 and Table 7) indicates, that the best two marketing science diffusion models are the Pearl logistic and the Bass. Across Horizons One to Three, they are the best and are narrowly beaten by the Gompertz at Horizon Four. There is little to choose between those two; they are about 16 percent and 15 percent MAPE respectively at Horizon One and grow to 39 percent and 40 percent MAPE respectively at Horizon Four. The Gompertz and the log-logistic are the two worst marketing science models. The Gompertz improves from about 4.5 percent MAPE poorer than the best at Horizon One to having slightly better MAPE to the best by Horizon Four (about 3.5 percent MAPE better) in a smooth progression of improvement. The log-logistic on the other hand, deteriorates from near identical MAPE to the best models at Horizons One and Two to about 10 percent MAPE inferior to the best by Horizon Four. The Gompertz and log-logistic MAPE performances’ cross over at about half way between Horizons Two and Three where both are inferior to the two best models by about 4-5 percent MAPE. The straight-line model is the best predictor at Horizon One and matches the best two diffusion models at Horizon Two. For later horizons, the superior match of the marketing science models to the decline curve shape, shows in three models; with the Pearl logistic, the Gompertz, and Bass models all being better than the straight-line model. The straight-line model’s forecast errors, as expected, trace an S-curve of MAPE reflecting how it is not a realistic model of the decline data generating process, despite its competitive MAPE levels.
Figure 12. Mean forecast MAPE of the marketing science models.
Table 7
Table 7. Mean Forecast MAPE of Marketing Science Models
Mean Forecast MAPE of Marketing Science Models
In summary, all models including the straight-line model, have deteriorating MAPE as the horizons extend, and the straight-line deteriorates quicker, because it is not reflecting S-curve decline, the typical pattern in the data. As the MAPE measure is burdened with expressing the errors as a percentage of a rapidly falling variable of interest, it tends to overemphasise error levels at the expense of an understanding of a more general understanding of forecast performance of the model. In the next section, the relative performance of the models is investigated, using the naïve benchmark, based relative measures.
7.4.4.
Model relative forecast accuracy
The primary performance measure in this study is the relative measure CumRAE, a related but bounded variant published during this study is also utilised to explore the their relative performance, when faced with various levels of variation in error range (Chen et al., 2017) see section 6.3.2. Table 8 shows the percentage difference between the two relative error measures. As can be seen, these differences are mostly small except in the case of the performance of the straight-line model, which generated sufficiently large errors at longer horizons to produced large differences between CumRAE and UMBRAE see Table 8. It indicates that there are outliers in error magnitude generated by the straight-line model forecasts, which are being removed by the bounding process in the UMBRAE measure. However, if the aim was to test the forecast capability of a model one might well be interested in seeing the performance when dealing with outliers (poorer forecast) and CumRAE provides that when the Winsorising process is omitted from data preparation. However, to understand the effect those outlier forecasts they would need to be removed in some way. This could be done by deleting forecasts, Winsorising the error data or undertaking a comparison with UMBRAE, making UMBRAE useful to a forecast research in ways not envisaged by Chen et al. (2017), and arguably easier to execute than the Winsorising process which involves trimming the data.
Horizon Pearl logistic Gompertz Bass Log-logistic Straight-line
% % % % %
1 16.54 22.04 15.72 16.54 12.28
2 20.95 24.94 19.94 21.92 20.99
3 28.55 30.06 27.85 32.90 43.88
4 39.35 36.81 40.31 51.01 50.64
Figure 13 and Figure 14 illustrate the virtually identical results for diffusion models with those two measures. Table 9 and Table 10 list the CumRAE and UMBRAE for the models’ forecast performance. Because of this strong similarity of result, they are reported together and any differences are analysed at the end of this section.
Table 8
Table 8. Absolute Percentage Difference of Mean CumRAE and Mean UMBRAE for Marketing Science Models
Absolute Percentage Difference of Mean CumRAE and Mean UMBRAE for Marketing Science Models
Measuring relative accuracy with CumRAE and UMBRAE, the three better models were the log-logistic, the Bass and the Pearl logistic. Their performance was closely grouped, with the Pearl logistic the worst of that group, at early horizons, being worse that the CumRAE benchmark at Horizons One (CumRAE 1.04) and only being about fourteen percent better than the naïve benchmark (CumRAE 0.86) at Horizon Two. The log-logistic was the best model at the first two horizons and bettering the naïve benchmark by 20 percent and 30 percent respectively over those horizons. Over those same horizons, the Bass was halfway between. By the last two horizons, the log- logistic and the Pearl logistic had swapped positions of best and worst. By Horizon Four, the three models had remarkable similar performance with the log-logistic performing the worst with performance 45 percent better than the naïve benchmark. The Pearl logistic was the best performing of all models at that horizon (55 percent better). The Bass model continued to be placed between those two models. Of the four marketing science models, the worst performing was the Gompertz that performed substantially worse than the naïve benchmark measure over Horizons One and Two, and at no time bettered any of the other models. The straight-line is the best predictor overall with performance about 67 percent better than the naïve benchmark across all horizons. The straight-line model’s performance is critically, linked to the location of the fixed origin for the forecasts at, or just before, the 50 percent share fall point. This is because this is an area where the rate of change in decline rate is at its lowest (the curve is most linear like).
Pearl logistic Gompertz Bass Log-logistic Straight-line
Horizon Mean SD Mean SD Mean SD Mean SD Mean SD
1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
2 2.71 1.92 2.35 1.55 2.02 1.73 2.45 1.11 2.57 0.60
3 2.30 2.43 4.05 3.75 2.03 1.88 2.44 0.20 8.89 8.19
4 2.50 3.01 1.45 3.52 1.95 2.82 3.30 1.36 15.31 13.84
Note: Number of forecasts for each model horizon are H1-H2 n = 25, H3 n= 22, H4 n= 20 Note: Percentage difference calculated with respect to CumRAE
Figure 13. Mean CumRAE of marketing science models.
Figure 14. Mean UMBRAE of marketing science models.
Table 9
Table 9. Mean CumRAE of Marketing Science Models
Mean CumRAE of Marketing Science Models
Table 10
Table 10. Mean UMBRAE of Marketing Science Models
Mean UMBRAE of Marketing Science Models
In summary, at no time was the Gompertz competitive with any of the models, and would have been rejected as a suitable model based on the CumRAE and UMBRAE measures, even though it beat the naïve benchmark in the last two horizons. At no time did the diffusion models beat the straight-line model. The three better performing diffusion models, perform
Pearl logistic Gompertz Bass Log-logistic Straight-line
Horizon Mean SD Mean SD Mean SD Mean SD Mean SD
1 1.04 1.57 1.37 1.87 0.90 1.28 0.80 0.89 0.35 0.33
2 0.86 1.39 1.17 1.71 0.77 1.14 0.69 0.80 0.36 0.32
3 0.50 0.52 0.71 0.66 0.53 0.63 0.56 0.52 0.39 0.33
4 0.45 0.49 0.68 0.63 0.50 0.59 0.55 0.48 0.44 0.39
Note: Normal distribution plots of the CumRAE and UMBRAE indicated that outcomes were not normal; despite this and as is common practice the standard deviation (SD) dispersion statistic is presented along with the mean.
Note: number of forecasts for each model horizon are H1-H2 n = 25, H3 n= 22, H4 n= 20
Pearl logistic Gompertz Bass Log-logistic Straight-line
Horizon Mean SD Mean SD Mean SD Mean SD Mean SD
1 1.04 1.57 1.37 1.87 0.90 1.28 0.80 0.89 0.35 0.33
2 0.88 1.42 1.20 1.74 0.78 1.16 0.71 0.81 0.35 0.32
3 0.51 0.54 0.74 0.68 0.54 0.64 0.57 0.52 0.36 0.30
4 0.46 0.51 0.69 0.65 0.51 0.61 0.56 0.49 0.37 0.34
Note: Normal distribution plots of the CumRAE and UMBRAE indicated that outcomes were not normal; despite this and as is common practice the standard deviation (SD) dispersion statistic is presented along with the mean.
such that you would choose the log-logistic if you were interested in early horizons, the Pearl logistic if you were interested in later horizons and the Bass if you needed the best performer across the horizons on average. If the length of the series limited you to forecasting after getting to this 50 percent fall point then the best model by a substantial margin is the straight- line drawn through the last three points prior to the 50 percent point. It is interesting to note the swapping of ranking of the two best models from that provided by MAPE to that provided by these two measures. This is difficult to explain simply but is most likely related to three things; The difference in sensitivity to scale between MAPE which is highly sensitive as described in section 6.3.1, and the two relative measures which are largely insensitive. The scale issue is related to the generation of errors, and the level of the actual data. When data is falling away then a given magnitude of effort becomes a larger percentage error. Also in data with substantial changes in both scale and rate of change (as are decline curves), then a fitted model will generate a wide error distribution in terms of magnitude variation. These three factors drive the different rankings from the different measures.
7.4.5.
Model forecast bias
Sometimes the focus of interest is the direction of the error in forecasts. Figure 15 and Table 11 present the mean error in market share terms. Table 11, also includes the standard deviations of the errors in prediction. Those errors demonstrate that the Bass model forecasts close to neutral performing best, indicating little bias over many series and across all horizons. The Bass model performs best with on average only one percent overestimation of market share, that is, predicts more rapid decline than the actual market. The standard deviation of the Bass estimates are about 11 percent market share across all horizons. The Pearl logistic, the next best model, also overestimates decline, but by between two and five percent as the horizon extends and has a worst case standard deviation of eleven percent market share. The straight-line model underestimates over the first three horizons and over estimates at Horizon Four, and is competitive with the best two target diffusion models and betters the Pearl logistic model across the last three horizons. However, its performance is be deteriorating rapidly by Horizon Four. The log-logistic performs competitively with the Pearl logistic in terms of error level at early horizons but deteriorates to be the third worst at Horizons Three and Four, it underestimates rather than over estimates as do the other models. On this measure, the Gompertz performs very poorly and would be rejected.
Figure 15. Mean error of forecast share for the marketing science models.
Table 11
Table 11. Mean Share Error of Forecast of the Marketing Science Models
Mean Share Error of Forecast of the Marketing Science Models
Note: number of forecasts for each model horizon are H1-H2 n = 25, H3 n= 22, H4 n= 20
In summary, the size of the mean errors and the standard deviation of errors (see Table 11) indicate that the mean forecast error of the models appear to give good (relatively unbiased) forecasts.
7.4.6.
Model forecast variability
Taking the perspective that an adequate level of relative accuracy is achieved, and then the focus might naturally fall to the consistency of the method in reaching that level. Two ways to express forecast variability are explored. The first is the range of variation that might occur in an estimate based on the probability distribution for a normally distributed sample. The second is the proportion of times a models forecasts better than a benchmark. Both methods are used to identify model superiority.
Pearl logistic Gompertz Bass Log-logistic Straight-line
Horizon Mean SD Mean SD Mean SD Mean SD Mean SD
1 -0.02 0.09 -0.06 0.11 0.00 0.10 0.03 0.09 0.03 0.08
2 -0.04 0.11 -0.09 0.12 -0.01 0.11 0.04 0.10 0.03 0.10
3 -0.03 0.12 -0.11 0.12 -0.01 0.11 0.06 0.11 0.02 0.13
7.4.6.1. Parametric estimation of the forecast interval
Data error distribution normality for models was indicated across the 25 series, at all horizons, with stronger normality indicated at longer horizons. The Shapiro Wilks (1965) test used is the most powerful normality test but like all such tests is weaker in power on small samples (Razali & Wah, 2011). Differences between the mean and the median at some model/horizon estimates indicated the weakness of the test on this sample. However, assuming error distribution normality, parametric estimation of forecast uncertainty were calculated as described in section 6.3.5.
Charting forecast estimation intervals via fan charts assists in demonstrating the potential variation in a model’s forecast. An example of this at 80 percent and 95 percent confidence levels with respect to the mean estimate error is provided in Figure 16. The values for all model forecast intervals are presented in Table 12 and Table 13. Presenting error (bias) and forecast interval together provides a summary of the overall performance of models, and answers the question for managers: How good will that forecast likely to be? For example, it demonstrates that the Pearl logistic, Bass, straight-line models all exhibit low bias, and that the predictions for the Gompertz and the log-logistic are highly biased, and thus the range of the estimate intervals also represent strong bias in the forecast. Managers are often interested in the risks and opportunities at those outer limits of the estimate, and this presentation shows the limits of potential forecasts. However, as a way to present the relative variability performance of each of the models it fails because it places the interval on a biased origin making interpretation of relative intermodal interval challenging.
Table 12
Table 12. The 80 Percent Confidence Bound for the Mean Value of the Marketing Science Model Forecasts
The 80 Percent Confidence Bound for the Mean Value of the Marketing Science Model Forecasts
Table 13
Table 13. The 95 Percent Confidence Bound for the Mean Value of the Marketing Science Model Forecasts
The 95 Percent Confidence Bound for the Mean Value of the Marketing Science Model Forecasts.
Note: number of forecasts for each model horizon are H1-H2 n = 25, H3 n= 22, H4 n= 20
So a presentation based on a zero biased forecast is required, to demonstrate the relative variation performance of the models. The relative size of the models’ parametrically estimated 80 percent forecast intervals (with respect to an unbiased forecast) are recorded in Figure 17 and Table 14. Here we can see how the marketing science models have very similar size estimated forecast intervals at each of the four horizons. Interestingly, the Bass scribes a different shape curve to the others, one more linear that the other marketing science models illustrating its consistency of performance across horizons. Importantly from a managerial point of view, the marketing science models have very similar variability in forecast performance.
Pearl logistic Gompertz Bass Log-logistic Straight-line
Horizon Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd 1 -0.15 0.10 -0.21 0.09 -0.14 0.13 -0.09 0.14 -0.07 0.13 2 -0.17 0.10 -0.25 0.06 -0.15 0.13 -0.10 0.18 -0.10 0.17 3 -0.19 0.12 -0.27 0.05 -0.16 0.14 -0.08 0.20 -0.16 0.20 4 -0.20 0.10 -0.28 0.02 -0.18 0.14 -0.08 0.21 -0.23 0.16
Note: number of forecasts for each model horizon are H1-H2 n = 25, H3 n= 22, H4 n= 20
Pearl logistic Gompertz Bass Log-logistic Straight-line
Horizon Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd 1 -0.22 0.17 -0.30 0.17 -0.21 0.20 -0.16 0.21 -0.12 0.19 2 -0.25 0.18 -0.33 0.15 -0.23 0.21 -0.17 0.25 -0.18 0.24 3 -0.28 0.21 -0.36 0.14 -0.25 0.23 -0.15 0.28 -0.26 0.30 4 -0.29 0.19 -0.36 0.10 -0.27 0.23 -0.16 0.30 -0.34 0.27
Figure 17. Relative size: 80 percent confidence intervals for marketing science models.
Table 14
Table 14. Relative Size of Forecast Intervals (80 Percent) for Marketing Science Model Forecasts
Relative Size of Forecast Intervals (80 Percent) for Marketing Science Model Forecasts
Note: number of forecasts for each model horizon are H1-H2 n = 25, H3 n= 22, H4 n= 20
Overall, there is little to separate the marketing science models’ forecast intervals. The straight-line model is better earlier and deteriorates to be worse than the marketing science models in later horizons. Therefore, in summary the marketing science models demonstrate uniform level of consistency while the poor ability of the straight-line to replicate the data generating process of decline means its variance increases with horizon length.
7.4.6.2. Model forecast proportion of times better than benchmark
Another way to consider forecast consistency is to assess what proportion of time one method’s forecasts are better than a benchmark. Figure 18 and Table 15 demonstrate how frequently each model forecasts better than the naïve benchmark. With regard to bettering the naïve benchmark, the straight-line model is consistently better than the other models, across horizons. The log-logistic is the next most consistent model, bettering or equalling all models except the straight-line across all horizons. The Pearl logistic is worse earlier in
Pearl logistic Gompertz Bass Log-logistic Straight-line
Horizon Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd Lwr- Bnd Upr- Bnd 1 -0.12 0.12 -0.15 0.15 -0.13 0.13 -0.12 0.12 -0.10 0.10 2 -0.14 0.14 -0.15 0.15 -0.14 0.14 -0.14 0.14 -0.14 0.14 3 -0.16 0.16 -0.16 0.16 -0.15 0.15 -0.14 0.14 -0.18 0.18 4 -0.15 0.15 -0.15 0.15 -0.16 0.16 -0.15 0.15 -0.19 0.19
the horizons and better in later horizons than the Bass. The worst performer, the Gompertz, is the worst performer at all horizons, and at its best only beats the naïve benchmark 81percent of the time and then only at Horizon Three.
That the straight-line model is substantially more consistent in bettering the naïve benchmark than the others is interesting. It betters the scores of all models, at all horizons. In the first two horizons, it is substantially better. The log-logistic is consistently the best of the marketing science models, only being matched by the Pearl logistic in the fourth horizon.
Figure 18. Proportion of times marketing science models are better than the naïve benchmark.
Table 15
Table 15. Proportion of times Marketing Science Models are better than the Naïve Benchmark
Proportion of times Marketing Science Models are better than the Naïve Benchmark
Summary
This study investigated the performance of functional forms developed for modeling diffusion when applied to forecasting decline. The Bass and the Logistic model are the best performing of the marketing science models. Models that performed well had the same
Pearl logistic Gompertz Bass Log-logistic Straight-line
Horizon Cum- RAE UMB- RAE Cum- RAE UMB- RAE Cum- RAE UMB- RAE Cum- RAE UMB- RAE Cum- RAE UMB- RAE 1 0.72 0.72 0.68 0.68 0.76 0.76 0.84 0.84 0.96 0.96 2 0.76 0.76 0.72 0.72 0.84 0.84 0.88 0.88 0.92 0.92 3 0.86 0.86 0.86 0.82 0.86 0.86 0.91 0.91 0.95 0.95 4 0.90 0.90 0.85 0.85 0.85 0.85 0.90 0.90 0.90 0.95
results with CumRAE and UMBRAE indicating they dealt with outlier curve shapes better than the weaker performing models. Interestingly the worst performing models registered better UMBRAE than CumRAE figures indicating their poor performance on outlier curve shapes, was an issue.
The log-logistic and the Gompertz appear to be weak in comparison to the Bass and Pearl