Del acto a la posibilidad

The surge of connectionist techniques, such as multilayer perceptrons, has created new possibilities for the field of forecasting. Currently, the application of these methods requires some effort from an analyst, in processes such as data analysis and model selection. In this chapter, an adaptive approach is presented, assuming no prior knowledge over each series. Moreover, the proposed system works automatically, performing model selection on the fly, being able to choose among distinct multilayer perceptrons, ranging from linear models to complex nonlinear ones. However, this added autonomy has the handicap of increasing the computational complexity.

Comparative experiments, among conventional (e.g., Holt-Winters & Box-Jenkins) and connectionist approaches, with several real and artificial series from different domains, were held. These have shown that the Holt-Winters method, although very simple,

presents a good performance on linear series with seasonal and trended components. However, when the domain gets more complex, with nonlinear behavior, the traditional methods are clearly not appropriate. The proposed neural approach shows its strength exactly in these scenarios.

On the other hand, it was possible to confirm that neural networks, such as the multilayer perceptrons,are indeed very powerful tools for regression tasks, although they rely heavily on the network design. Poor structures provide insufficient learning capabilities, while ones that are too complex lead to overfitting. One promising possibility for topology selection is to use a hybrid combination of evolutionary and neural procedures. Indeed, the genetic algorithm, here taken as the main engine, proves to be a powerful model selection tool. Regarding the evaluation of the neural forecasting generalization capabilities, the BIC statistic, which penalizes complexity, has revealed itself as an adequate solution. This criterion also presented the advantage of demanding a simple computation while increasing the selection pressure in favor of simpler structures, which are more comprehensible.

In the future, it is intended to apply the evolutionary neural networks to other forecasting types, such as long-term, multivariate, or real-time prediction, in real-world applica- tions (e.g., bioengineering or Internet traffic). Although the proposed methodology obtained interesting results in the forecasting arena, it is potentially useful in other domains where multilayer perceptrons can be applied (e.g., classification tasks). Finally, it also is aimed to explore the evolutionary optimization with different neural architectures (e.g., recurrent neural networks).

References

Basu, S., & Mukherjee, A. (1999). Time series models for Internet traffic. Proceedings of the 24th _{Conf. on Local Computer Networks (pp. 164-171).}

Box, G., & Jenkins, G. (1976). Time series analysis: Forecasting and control. San Francisco: Holden Day.

Cao, L., & Tay, F. (2003). Support vector machine with adaptive parameters in financial time series forecasting. IEEE Transactions on Neural Networks, 14(6), 1506-1518. Chatfield, C. (1993). Neural networks: Forecasting breakthrough or passing fad? Inter-

national Journal of Forecasting, 9, 1-3.

Chen, S., & Lu, C. (1999). Would evolutionary computation help in designs of ANNs in forecasting foreign exchange rates? Proceedings of the 1999 Congress on Evolu- tionary Computation (vol.1, pp. 267-274).

Cortez, P., Allegro, F., Rocha, M., & Neves, J. (2002). Real-time forecasting by bio- inspired models. In M. Hamza (Ed.), Proceedings of the 2nd International Confer- ence on Artificial Intelligence and Applications (AIA), Málaga, Spain (pp. 52-57). Cortez, P., Rocha, M., Machado, J., & Neves, J. (1995). A neural network based forecasting system. Proceedings of the ICNN’95 - IEEE Int. Conf. on Neural Networks, Perth, Western Australia (vol. 5, pp. 2689-2693).

Cortez, P., Rocha, M., & Neves, J. (2004). Evolving time series forecasting ARMA models. Journal of Heuristics, 10(4), 415-429.

Falco, I., Cioppa, A., Iazzetta, A., Natale, P., & Tar, E. (1998). Optimizing neural networks for time series prediction. Proceedings of the Third World Conference on Soft Computing (WSC3).

Faraday, J., & Chatfield, C. (1998). Times series forecasting with neural networks: A case study. Applied Statistics, 47, 231-250.

Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). The KDD process for extracting useful knowledge from volumes of data. Communications of ACM, 39(11), 27-34. Flexer, A. (1996). Statistical evaluation of neural networks experiments: Minimum require- ments and current practice. Proceedings of the 13th European Meeting on Cybernetics and Systems Research, Vienna, Austria (vol. 2, pp. 1005-1008). Grau, F. (1992). Genetic synthesis of boolean neural networks with a cell rewriting

developmental process. In D. Whitley & J. Schaffer (Eds.), Proceedings of COGANN, Los Alamitos, CA (pp. 55-74). IEEE.

Hanke, J., & Reitsch, A. (1989). Business forecasting. MA: Allyn and Bancon. Harvey, A. (1989). Forecasting, structural time series models and the Kalman filter.

Cambridge: C. U. P.

Haykin, S. (1999). Neural networks: A comprehensive foundation. Prentice-Hall. Holland, J. (1975). Adaptation in natural and artificial systems. Unpublished doctoral

thesis, University of Michigan, Ann Arbor.

Huang, W., Xu, B., & Chan-Hilton, A. (2004). Forecasting flows in Apalachicola River using neural networks. Hydrological Processes, 18, 2545-2564.

Hyndman, R., & Akram, M. (2005). Time series data library. Retrieved from www- personal.buseco.monash.edu.au/~hyndman/TSDL/

Kaboudan, M. (2003). Forecasting with computer-evolved model specifications: A genetic programming application. Computers & Operations Research, 30, 1661- 1681.

Kwok, T., & Yeung, D. (1999). Constructive algorithms for structure learning in feedforward neural networks for regression problems: A survey. IEEE Transactions on Neural Networks, 8(3), 630-645.

Lapedes, A., & Farber, R. (1987). Non-linear signal processing using neural networks: Prediction and system modelling. Tech. Rep. LA-UR-87-2662, Los Alamos Na- tional Laboratory.

Makridakis, S., Wheelwright, S., & Hyndman, R. (1998). Forecasting: Methods and applications (3rd _{ed.). John Wiley & Sons.}

Michalewicz, Z. (1996). Genetic algorithms + data structures = evolution programs (3rd

ed.). Springer-Verlag.

Neves, J., Rocha, M., Rodrigues, H., Biscaia, M., & Alves, J. (1999). Adaptive strategies and the design of evolutionary applications. In W. Banzhaf et al. (Eds.), Proceed- ings of the Genetic and Evolutionary Computation Conference, Orlando, FL (pp. 473-479). Morgan Kaufmann.

Opitz, D., & Shavlik, J. (1994). Using genetic search to refine knowledge-based neural networks. In W. Cohen & H. Hirsh (Eds.), Proceedings of the 11th International Conference on Machine Learning. San Francisco: Morgan Kaufmann.

Peitgen, H., Jürgens, H., & Saupe, D. (1992). Chaos and fractals: New frontiers of science. New York: Springer-Verlag.

Riedmiller, M. (1994). Supervised learning in multilayer perceptrons: From backpropagation to adaptive learning techniques. Computer Standards and Interfaces, 16.

Rocha, M., Cortez, P., & Neves, J. (2003). Evolutionary neural network learning. In F Pires & S. Abreu (Eds.), Progress in artificial intelligence, LNAI 2902 (pp. 24-28). Springer.

Rycroft, R. (1993). Microcomputer software of interest to forecasters in comparative review. International Journal of Forecasting, 531-575.

Sarle, W. (1995). Stopped training and other remedies for overfitting. Proceedings of the 27th Symposium on the Interface of Computer Science and Statistics (pp. 352-360). Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6, 461-

464.

Setiono, R. (2003). Techniques for extracting classification and regression rules from artificial neural networks. In D. Fogel & C. Robinson (Eds.), Computational intelligence: The experts speak (pp. 99-114). IEEE Press/Wiley Interscience. Shoneburg, E. (1990). Price prediction using neural networks: A project report.

Neurocomputing, 2, 17-27.

Siddiqi, A., & Lucas, S. (1998). A comparison of matrix rewriting versus direct encoding for evolving neural networks. Proceedings of the IEEE Int. Conf. on Evolutionary Computation (pp. 392-397). Piscataway, NJ: IEEE Press.

Tang, Z., & Fishwick, F. (1993). Feed-forward neural nets as models for time series forecasting. ORSA Journal of Computing, 5(4), 374-386.

Thimm, G., & Fiesler, E. (1995). Evaluating pruning methods. Proceedings of the International Symposium on Artificial Neural Networks, Taiwan, Republic of China (pp. 20-25).

Tong, H. (1983). Threshold models in non-linear time series analysis. Springer lecture notes in statistics 21. Springer-Verlag.

Ulbricht, C. (1994). Multi-recurrent networks for traffic forecasting. Proceedings of AAAI’94 Conference, Seattle, WA.

Weigend, A., & Gershenfeld, N. (1994). Time series prediction: Forecasting the future and understanding the past (pp. 3rd _{ed.). Addison-Wesley.}

Winters, P. (1960). Forecasting sales by exponentially weighted moving averages. Management Science, 6, 324-342.

Yao, X. (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87(9), 1423- 1447.

Chapter IV

Development of