CONDICIONES GENERALES
Artículo 7. Fachadas y particiones 7.1 Fachadas de fábrica
7.1.1 Fachadas de piezas de arcilla cocida y de hormigón Descripción
symbols, letr denote the maximum number of transitions to any state over all
input symbols ofM. Furthermore, let min and
max denote the minimum and
maximum, respectively, of all transitions weights ijkin
M. Then, a sparse recur-
rent neural network withnstate andminput neurons can be constructed fromM
such that the internal state representation remains stable if (1)[ f ] 1 < 1 r max ( 1 2 + min [ 0 f ] 1 H ), (2)[ + h ] 1 > 1 min ( 1 2 + max [ f ] 1 + [ 0 h ] 1 H ), (3)H>max(H 0 (r);H + 0 (r)).
Furthermore, the constructed network has at most3mnsecond-order weights
with alphabet w
=f H;0;+Hg,n+1biases with alphabet b
=f H=2g,
For min
= max
=1, conditions (1)-(3) of the above theorem reduce to those found for stable DFA encodings [Omlin, 1996]. This is consistent with a crisp representation of DFA states.
VII.
SIMULATIONS
In order to test our theory, we constructed a fuzzy encoding of a randomly generated FFA with 100 states (after the execution of the FFA transformation al- gorithm) over the input alphabetf0;1g. We randomly assigned weights in the range[0;1]to all transitions in increments of 0.1. The maximum indegree was D
in
(M)=r= 5. We then tested the stability of the fuzzy internal state repre- sentation on 100 randomly generated strings of length 100 by comparing, at each time step, the output signal of each recurrent state neuron with its ideal output sig- nal (since each recurrent state neuronS
icorresponds to a FFA state q
i, we know the degree to whichq
iis occupied after input symbol a
khas been read: either 0or
ijk). A histogram of the differences between the ideal and the observed signal of state neurons for selected values of the weight strengthH over all state neurons and all tested strings is shown inFigure 11. As expected, the error decreases for increasing values ofH. We observe that the number of discrepancies between the desired and the actual neuron output decreases ‘smoothly’ for the shown values of H (almost no change can be observed for values up toH = 6). The most significant change can be observed by comparing the histograms forH = 9:7 andH = 9:75: The existence of significant neuron output errors forH = 9:7 suggests that the internal FFA representation is highly unstable. ForH 9:75, the internal FFA state representation becomes stable. This discontinuous change can be explained by observing that there exists a critical valueH
0
(r)such that the number of stable fixed points also changes discontinuously from one to two for H <H
0
(r))andH>H 0
(r)), respectively (seeFigure 11). The ‘smooth’ transi- tion from large output errors to very small errors for most recurrent state neurons (Figure 11a-e) can be explained by observing that not all recurrent state neurons receive the same number of residual inputs; some neurons may not receive any residual input for some given input symbola
kat time step
t; in that case, the low signals of those neurons are strengthened tog(0;~ H;
i:k
) ' 0(note that strong low signals imply strong high signals by Lemma 6.7).
VIII.
CONCLUSIONS
Theoretical work that proves representational relationships between differ- ent computational paradigms is important because it establishes the equivalences of those models. Previously it has been shown that it is possible to determin- istically encode fuzzy finite state automata (FFA) in recurrent neural networks by transforming any given FFA into a deterministic acceptor which assign string membership [Omlin, 1998]. In such a deterministic encoding, only the network’s classification of strings is fuzzy, whereas the representation of states is crisp. The correspondence between FFA and network parameters - i.e., fuzzy transition mem-
(a) (b) (c) (d) (e) (f) 0.2e+5 0.4e+5 0.6e+5 0.8e+5 1.0e+5 1.2e+5 1.4e+5 1.6e+5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Frequency
Absolute Neuron Output Error
0 0.2e+5 0.4e+5 0.6e+5 0.8e+5 1.0e+5 1.2e+5 1.4e+5 1.6e+5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Frequency
Absolute Neuron Output Error 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Frequency
Absolute Neuron Output Error
0 0.5e+5 1.0e+5 1.5e+5 2.e+5 2.5e+5 3.0e+5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Frequency
Absolute Neuron Output Error
0 0.5e+5 1.0e+5 1.5e+5 2.0e+5 2.5e+5 3.0e+5 3.5e+5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Frequency
Absolute Neuron Output Error
0 0.5e+5 1.0e+5 1.5e+5 2.0e+5 2.5e+5 3.0e+5 3.5e+5 4.0e+5 4.5e+5 5.0e+5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Frequency
Absolute Neuron Output Error
0 2.0e+5 4.0e+5 6.0e+5 8.0e+5 10.0e+5 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Frequency
Absolute Neuron Output Error
Figure 11. Stability of FFA State Encoding: The histograms shows the absolute neuron output error of a network with 100 neurons that implements a randomly generated FFA and reads 100 randomly generated strings of length 100 for different values of the weight strengthH. The error was computed by comparing, at each time step, the actual with the desired output of each state neuron. The distribution of neuron output signal errors are for weight strengths (a)H =6:0, (b)H =9:0, (c)H = 9:60, (d)H = 9:65, and (e)
Here, we have demonstrated analytically and empirically that it is possible to encode FFA in recurrent networks without transforming them into deterministic acceptors. The constructed network directly represents FFA states with the de- sired fuzziness. That representation requires (1) a slightly increased functionality of sigmoidal discriminant functions (it only requires the discriminants to accom- modate variable output range), and (2) a transformation of a given FFA into an equivalent FFA with a larger number of states. (We have found empirically that the increase in automaton size is roughly proportional toNKwhereN andK are the automaton and alphabet size, respectively.) In the proposed mapping FFA !recurrent network, the correspondence between FFA and network parameters remains intact; this can be significant if the physical properties of some unknown dynamic, nonlinear system are to be derived from a trained network modeling that system. Furthermore, the analysis tools and methods used to demonstrate the sta- bility of the crisp internal representation of DFA carried over and generalized to show stability of the internal FFA representation.
We speculate that other encoding methods are possible and that it is an open question as to which encoding methods are better. One could argue that, from a engineering point of view, it may seem more natural to use radial basis functions to represent fuzzy state membership (they are often used along with triangular and trapezoidal membership functions in the design of fuzzy systems) instead of sigmoidal discriminant functions (DFA can be mapped into recurrent neural networks with radialbasis functions [Frasconi, 1996]). It is an open question how mappings of FFA into recurrent neural networks with radial basis discriminant functions would be implemented and how such mappings would compare to the encoding algorithm described in this work.
The usefulness of training recurrent neural networks with fuzzy state repre- sentation from examples to behave like a FFA - the variable output rangecan be treated as a variable parameter and an update rule similar to that for network weights can be derived - and whether useful information can be extracted from trained networks has yet to be determined. In particular, it would be interesting to compare training and knowledge representation of networks whose discriminant functions have fixed and variable output ranges, respectively. Discriminant func- tions with variable neuron output range may open the door to novel methods for the extraction of symbolic knowledge from recurrent neural networks.
IX.
ACKNOWLEDGMENTS
We would like to acknowledge useful discussions with K. Bollacker, D. Hand- scomb and B.G. Horne and suggestions from the referees.
REFERENCES
Akers, L., Ferry, D., Grondin, R., Synthetic neural systems in VLSI. In An Intro-
Ashar, P., Devadas, S., Newton, A., Sequential Logic Synthesis. Kluwer Aca- demic Publishers, Norwell, 1992.
Berenji, H., Khedkar, P., Learning and fine tuning fuzzy logic controllers through reinforcement. IEEE Transactions on Neural Networks, (3)5, 724, 1992.
Bezdek, J., Fuzzy logic and neural networks. IEEE Transactions on Neural Net-
works, 3, 1992. Special Issue.
Bishop, C., Neural Networks for Pattern Recognition. Oxford University Press, 1995.
Blanco, A., Delgado, M., Pegalajar, M., Fuzzy grammar inference using neural networks. Tech. Rep., Department of Computer Science and Artificial Intelli- gence, University of Granada, Spain, 1997.
Bonissone, P., Badami, V., Chiang, K., Khedkar, P., Marcelle, K., Schutten, M., Industrial applications of fuzzy logic at General Electric. Proceedings of the
IEEE, (83)3, 450, 1995.
Bookman, L., Sun, R., Architectures for integrating symbolic and neural pro- cesses. Connection Science, 5(3,4), 1993. Special Issue.
Casey, M., The dynamics of discrete-time computation, with application to re- current neural networks and finite state machine extraction. Neural Computation, 8(6), 1135, 1996.
Cellier, F., Pan, Y., Fuzzy adaptive recurrent counterpropagation neural networks: A tool for efficient implementation of qualitative models of dynamic processes. J.
Systems Engineering, 5(4), 207, 1995.
Chiu, S., Chand, S., Moore, D., Chaudhary, A., Fuzzy logic for control of roll and moment for a flexible wing aircraft. IEEE Control Systems Magazine, 11(4), 42, 1991.
Cichocki, A., Unbehauen, R., Eds. Neural Networks for Optimization and Signal
Processing. John Wiley, New York, 1993.
Cleeremans, A., Servan-Schreiber, D., McClelland, J., Finite state automata and simple recurrent neural networks. Neural Computation, 1(3), 372, 1989.
Corbin, J., A fuzzy logic-based financial transaction system. Embedded Sys- tems Programming, 7(12), 24, 1994.
Dubois, D., Prade, H., Fuzzy Sets and Systems: Theory and Applications, Vol. 144 of Mathematics in Science and Engineering. Academic Press, 220, 1980. Elman, J., Finding structure in time. Cognitive Science, 14, 179, 1990.
Franquelo, L., Chavez, J., Fasy: A fuzzy-logic based tool for analog synthe- sis. IEEE Transactions on Computer-Aided Design of Integrated Circuits, 15(7), 705, 1996.
Frasconi, P., Gori, M., Maggini, M., Soda, G., Representation of finite state au- tomata in recurrent radial basis function networks. Machine Learning, 23, 5,
1996.
Fu, L.-M., Neural Networks in Computer Intelligence. McGraw-Hill, Inc., New York, 1994.
Gaines, B., Kohout, L., The logic of automata. International Journal of Gen-
eral Systems, 2, 191, 1976.
Giles, C., Gori, M., Eds. Adaptive Processing of Sequences and Data Struc- tures. Lecture Notes in Artificial Intelligence. Springer-Verlag, 1998.
Giles, C., Miller, C., Chen, D., Chen, H., Sun, G., Lee, Y., Learning and ex- tracting finite state automata with second-order recurrent neural networks. Neural
Computation, 4(3), 393, 1992.
Giles, C., Omlin, C., Extraction, insertion and refinement of symbolic rules in dynamically driven recurrent neural networks. Connection Science, 5(3,4), 307, 1993.
Giles, C., Sun, R., Zurada, J., Neural networks and hybrid intelligent models: Foundations, theory, and applications. IEEE Transactions on Neural Networks, 9(5), 721, 1998. Special Issue.
Grantner, J., Patyra, M., VLSI implementations of fuzzy logic finite state ma- chines. In Proceedings of the Fifth IFSA Congress, 781, 1993.
Grantner, J., Patyra, M., Synthesis and analysis of fuzzy logic finite state ma- chine models. In Proc. of the Third IEEE Conf. on Fuzzy Systems, I, 205, 1994. Hardy, T. L., Multi-objective decision-making under uncertainty fuzzy logic methods. Tech. Rep. TM 106796, NASA, Washington, D.C., 1994.
Haykin, S., Neural Networks, A Comprehensive Foundation. Prentice Hall, En- glewood Cliffs, NJ, 1998.
Hendler, J., Developing hybrid symbolic/connectionist models. In Advances
in Connectionist and Neural Computation Theory, Barnden, J., Pollack, J., Eds.
Ablex Publishing, 1991.
Herrmann, C., A hybrid fuzzy-neural expert system for diagnosis. In Proc.
of the Fourteenth International Joint Conf. on Artificial Intelligence, I, 494, 1995.
Honavar, V., Uhr, L., Eds., Artificial Intelligence and Neural Networks: Steps
toward Principled Integration. Academic Press, 1994.
Hopcroft, J., Ullman, J., Introduction to Automata Theory, Languages, and Com-
putation. Addison-Wesley Publishing Company, Inc., Reading, PA, 1979.
Kasabov, N., Foundations of Neural Networks, Fuzzy Systems, and Knowledge
Engineering. MIT Press, Cambridge, 1996.
Khalil, H., Nonlinear Systems. Macmillan Publishing Company, New York, 1992. Khan, E., Unal, F., Recurrent fuzzy logic using neural networks. In Advances
in fuzzy logic, neural networks, and genetic algorithms, Furuhashi, T., Ed., Lec-
ture Notes in AI. Springer-Verlag, 1995.
Kickert, W. J. M., van Nauta Lemke, H., Application of a fuzzy controller in a warm water plant. Automatica, 12(4), 301, 1976.
Kleene, S., Representation of events in nerve nets and finite automata. In
Automata Studies, Shannon, C., McCarthy, J., Eds. Princeton University Press,
Princeton, NJ, 3, 1956.
Kosmatopoulos, E., Christodoulou, M., Structural properties of gradient recur- rent high-order neural networks. IEEE Transactions on Circuits and Systems,
42(9), 592, 1995.
Kosmatopoulos, E., Christodoulou, M., Neural networks for identification of fuzzy dynamical systems: An application to identification of vehicle highway systems. In Proceedings of the 4th IEEE Mediterranean Symposium on New
Directions in Control and Automation, 23, 1996.
Kosmatopoulos, E., Christodoulou, M., Recurrent neural networks for approx- imation of fuzzy dynamical systems. International Journal of Intelligent Control
Kosmatopoulos, E., Polycarpou, M., Christodoulou, M., Ioannou, P., High-order neural networks for identification of dynamical systems. IEEE Transactions on
Neural Networks, 6(2), 422, 1995a.
Lee, C., Fuzzy logic in control systems: fuzzy logic controller. IEEE Trans- actions on Man, Systems, and Cybernetics, 20(2), 404, 1990.
Maclin, R., Shavlik, J., Using knowledge-based neural networks to improve al- gorithms: Refining the Chou-Fasman algorithm for protein folding. Machine Learning, 11, 195, 1993.
Mead, C., Analog VLSI and Neural Systems. Addison-Wesley, Reading, PA, 1989.
Mensch, S., Lipp, H., Fuzzy specification of finite state machines. In Proceedings
of the European Design Automation Conference, 622, 1990.
Minsky, M., Computation: Finite and Infinite Machines. Prentice-Hall, Inc., Englewood Cliffs, NJ, 1967.
Omlin, C., Giles, C., Constructing deterministic finite-state automata in recur- rent neural networks. Journal of the ACM, 43(6), 937, 1996.
Omlin, C., Giles, C., Extraction of rules from discrete-time recurrent neural net- works. Neural Networks, 9(1), 41, 1996a.
Omlin, C., Giles, C., Rule revision with recurrent neural networks. IEEE Trans-
actions on Knowledge and Data Engineering, 8(1), 183, 1996b.
Omlin, C., Giles, C., Stable encoding of large finite-state automata in recurrent neural networks with sigmoid discriminants. Neural Computation, 8(7), 675,
1996c.
Omlin, C., Thornber, K., Giles, C., Fuzzy finite-state automata can be deter- ministically encoded into recurrent neural networks. IEEE Transactions on Fuzzy
Systems, 6(1), 76, 1998.
Palaniswami, M., Attikiouzel, Y., Marks, R., Fogel, D., Eds. Computational Intelligence: A Dynamic System Perspective. IEEE Press, Piscataway, NJ, 1995.
Pappis, C., Mamdani, E., A fuzzy logic controller for a traffic junction. IEEE
Transactions on Systems, Man, and Cybernetics, 7(10), 707, 1977.
Pollack, J., The induction of dynamical recognizers. Machine Learning, 7(2/3), 227, 1991.
Santos, E., Maximin automata. Information and Control, 13, 363, 1968.
Sheu, B. J., Neural Information Processing and VLSI. Kluwer Academic Publish- ers, Boston, 1995.
Siegelmann, H., Sontag, E., On the computational power of neural nets. Journal
of Computer and System Sciences, 50(1), 132, 1995.
Siegelmann, H. T., Neural Networks and Analog Computation: Beyond the Turing
Limit. Birkhauser, Boston, 1999.
Sun, R., Learning, action, and consciousness: A hybrid approach towards model- ing consciousness. Neural Networks, 10(7), 1317, 1997.
Thomas, S. F., Fuzziness and Probability. ACG Press, Wichita, KS, 1995. Thomason, M., Marinos, P., Deterministic acceptors of regular fuzzy languages.
IEEE Transactions on Systems, Man, and Cybernetics, 3, 228, 1974.
Unal, F., Khan, E., A fuzzy finite state machine implementation based on a neural fuzzy system. In Proceedings of the Third International Conference on Fuzzy
Systems, 3, 1749, 1994.
Watrous, R., Kuhn, G., Induction of finite-state languages using second-order recurrent networks. Neural Computation, 4(3), 406, 1992.
Yang, X., Kalambur, G., Design for machining using expert system and fuzzy logic approach. Journal of Materials Engineering and Performance, 4(5), 599, 1995.
Zadeh, L., Fuzzy sets. Information and Control, 8, 338, 1965.
Zadeh, L., Fuzzy languages and their relation to human and machine intelligence. Tech. Rep. ERL-M302, Electronics Research Laboratory, University of Califor- nia, Berkeley, 1971.
Zeng, Z., Goodman, R., Smyth, P., Learning finite state machines with self- clustering recurrent networks. Neural Computation, 5(6), 976, 1993.