The design of an optimal filter is based on previous knowledge on the signal statistics, and the obtained solution is optimal only in the case when the filtered signal really has the assumed statistical properties. However, it is very often the case in practice that the statistical properties of the signal are not available or they are variable. In that case the statistical parameters change over time and it becomes very difficult to design an adequate optimal solution.
A convenient approach in the said situations is the design of the so-called self-adapting filters, known as the adaptive filters. One of the most important properties of the adaptive filters is their ability to adapt, which leads to the desired system.
Instead of having a system strictly defined in advance, adaptive filters utilize the information from their surroundings to set the value of their parameters. This implies that the value of the parameter changes, according to the filter output, through a feedback and a suitable criterion function. The properties of the feed-back directly influence the charge of the parameter values, through a system procedure that is denoted as the learning process or the training period. During the training the filter output changes in accordance to the reference signal, i.e. the error value decreases with the advancement of the training process.
The design of the adaptive algorithm is fundamental for this approach. The purpose of the adaptive algorithm is the observation of the environment and the adaptation of the filter transfer function in dependence on the observed changes.
The algorithm starts from the previously defined initial conditions, which do not have to be coordinated with the environment, and based on the momentary values of the input, the output and the reference signal it tends to find the solution for an optimal filter. In a stationary environment it is expected that the filter converges to an optimal filter, while in nonstationary environment it is expected that the filter will follow changes with time and modify its parameters accordingly. The adaptive filters do not use any prior knowledge about the statistical properties of the signal, but estimate them based on a realization of the process, i.e. using a suitable sequence of time samples of the signal. There are different ways to estimate these properties and to apply them in the algorithm itself. A consequence of this is the existence of different adaptive algorithms. The choice of an adaptive algorithm depends on a particular application of the adaptive filter, and some of the key parameters are the convergence speed, adaptation success, robustness to errors, ability to follow fast changes, numerical stability, computational complexity and possibility of practical implementation.
A block diagram of an adaptive filter is shown in Fig.1.8. The adaptive filter includes two processes: (1) filtering process, which generates an output signal according to he input signal; (2) adaptive process, whose role is to adjust the filter parameters, i.e. its transfer function, in order to obtain an adequate output signal.
The adaptation process is controlled by the error signal, which is also called residual, and which represents a measure of the adaptation of the filter parameters, i.e. it is the indicator of conformity between the output and the reference signal.
When estimating the filter parameters, one often uses squared error signal, or the mean square error (MSE) as the optimization criterion. In dependence on a par-ticular use of the adaptive filter, the measure of the adaptation success may be based on the value of the estimated filter parameters, on the filter output signal or on the error signal.
The most often utilized adaptive filter structures are the transversal structures (or FIR), owing to their unconditional stability and the relatively simple analysis of the properties of these filters. Although other structures also find wide use, the subject of this text is the analysis of the adaptive FIR filters in a nonstationary environment, as well as the possibility to increase their convergence speed.
Namely, the estimation of the parameters of the real systems of interest is as a rule followed by difficulties stemming from the inherent nonlinearity and/or nonsta-tionarity of the system, as well as from noisy observational measurements.
In signal processing, the statistical properties of the input and the reference signal determine the environment of the adaptive filter. Although most of the analyses of adaptive filters in the available literature are based on the assumption of a stationary environment, the application of adaptive filters is especially con-venient for nonstationary environments. Nonstationarity can be categorized with regard to the change of statistical properties of the input signal, the reference signal or both simultaneously. In this text we consider the cases when the nonstationary model is a consequence of the variation of the estimated parameters of the filter, since all of the quoted types of nonstationarity may be represented in this manner.
The most significant measures of the properties of the adaptive filter in such an environment are the time necessary for the convergence of the algorithm for the estimation of the filter parameters at the onset of nonstationary changes, and the achieved accuracy of the estimated parameters after the convergence has been reached. Due to the mutual incongruity of these two requirements, the standard adaptive algorithms, adequate for the estimation of parameters in stationary con-ditions, do not give satisfactory estimation of parameters. Basically these are algorithms with unlimited memory, which take into account all previous values of the analyzed signal and perform the estimation of parameters in the next moment.
The result of such an approach are estimations of average behavior of the process in the time interval under consideration. To analyze nonstationary signals it is necessary to utilize an algorithm with limited memory, which is solved by introducing a variable forgetting factor. By generating such a variable forgetting factor using residuals, it is possible to adequately follow both slow and abrupt
reference signal
output signal +
input signal Adaptive filter
error signal
+
parameter change Fig. 1.8 Block diagram of
an adaptive filter
changes of the time-variable system parameters, without at the same time sig-nificantly impairing the desired quality of estimation in stationary mode of operation.
Since FIR adaptive filters find very wide application, it is often necessary to model systems with a basically IIR with an FIR filter. The consequence of this approach is that the dimensions of the vector of the estimated parameters may be very large in order to achieve satisfactory characteristics of the modeled system.
Besides that, due to the increase of the dimensions of the vector of estimated parameters, the number of iterations necessary for the convergence of the algo-rithm for the filter parameter estimation also grows. This fact represents the motivation for the synthesis of adaptive algorithms with increased convergence speed. One of the ways to increase the convergence speed, based on the optimal design of the input signal, is called the D-optimality. The essence of the optimal experiment design is an adequate choice of variables included in the experiment, in such a manner that the experiment itself is maximally informative with regard to the desired application. Besides that, in the class of informative experiments, the property of some experiments is that the vector of estimated parameters, obtained based on a finite set of data, reaches the desired value much faster than the vector of estimated parameters obtained under some other conditions. By choosing a criterion function maximizing the information content of an equivalent experi-ment, in which the object is excited by specially designed sequences, and com-bining with the procedure for the estimation of the unknown filter parameters, one arrives to the solution for the generation of the optimal input signal.
Let us also remind that the investigation of parameter estimation in various models of real systems resulted in the development of a number of algorithms possessing theoretically optimal properties with regard to a chosen criterion. In a majority of the cases, the methods for parameter estimation are based on an a priori accepted assumption that the random processes in the system have a Gaussian distribution. Numerous practical examples, however, showed an insuf-ficient justifiability of such an assumption, especially if there are large realizations of random perturbations in the system. It was established that optimal estimation procedures, based on the Gaussian assumption, may be very sensitive to the deviation of the real distribution of the perturbation from the assumed normal distribution, which results in estimations with less than satisfactory quality in many applications. On the other hand, the appearance of impulse noise is often met in practice when processing speech signals, images, biomedical signals, as well as in the solution of communication problems. To solve these problems one may use robust methods, based on the development of stochastic procedures which would be efficient even in the solutions of incomplete a priori information about the perturbations in the system. Bearing in mind the typical applications [17–21], we analyzed separately the problem of robustification of adaptive algorithms with regard to the impulse noise at the system output. The presented analysis is based on the application of the methodology of approximate maximum likelihood, also called in literature the M-Huber’s robust estimation.