Tecnologías avanzadas para la eliminación de PFAS de las aguas residuales 11

Linear models as well as ANN are based on a mapping function y(x, w), which is governed by the parameters w adapted during the training phase over a subset of the available data called training set. Prediction over new data, called test or validation set, is then based only on the learned parameter w. On the other side there are other kind of models which makes of the training data (or of a part of it) also in the testing phase. Many linear parametric models can make use of kernelf unctions of the form κ(x, x⁰) = φ(x)^Tφ(x⁰) evaluated on the training data,

classification or regression problem. One of the most peculiar characteristics of the support vector machine is that it can provide good generalization performances despite the fact that they dont incorporate problem-domain knowledge. In the context of linear classification a binary classification problem can be defined as follows, given the set of pairs:

{(xi, ti)}i = 1, 2, · · · , N (4.13) where xi is the iih input and tithe corresponding binary target output, i.e. ti∈ {−1, 1} and

y(x) = w^Tφ(x) + b (4.14)

where φ(x) denotes a fixed feature space transformation, while b is the bias parameter. If the training dataset is linearly separable in the features space, thus it must exist at least a couple (w, b) such that y(x) in the form of 4.13 satisfies:

y(x_n) > 0, ∀t_n= +1 (4.15)

y(x_n) < 0, ∀t_n= −1 (4.16)

Among the possibly multiple solutions to the problem in Equations (4.15) and (4.16), SVM approach is to select the one that maximize the margin, that is the minimum distance between the decision boundary and any of the samples, as illustrated for the two dimensional case in figure 4.4.

The margin can be calculated as the orthogonal distance between the closest point of the dataset and the derived hyperplane, since we are considering just points correctly classified, that is where t_ny(x_n) > 0, the perpendicular distance between the hyperplane and point x_n is:

tny(xn)

kwk = tn(w^Tφ(xn) + b)

kwk (4.17)

therefore in order to maximize the margin the following equation has to be solved:

arg max

w,b

{ 1 kwkmin

n tn(w^Tφ(xn) + b)} (4.18)

Applying the transformation w → κw and b → κb distance to the separation hyperplane does not change, thus for the closest point to the separation hyperplane it is possible to set:

Figure 4.4: The figures shows the case of the separating hyperplane and relative margins for the two dimensional case. Solid and empty circles represent the two data clusters separated by the hyperplane.

tn(w^Tφ(xn) + b) = 1. (4.19)

The maximization problem in 4.18 is equivalent to minimize kwk⁻¹

This constraint problem can be solved introducing the Lagrange multipliers an ≥ 0 with n = 1, . . . , N , the Lagrangian function is thus defined as follows:

L(w, b, a) = 1

2kwk²−

N

X

n=1

a_n{t_n(w^Tφ(x_n) + b) − 1} (4.20)

Setting the derivates with respect to w and b equal to zero we obtain:

N

X

n=1

antnφ(xn) = w (4.21)

and

L(a) =˜

By means of the kernel functions κ the maximum margin approach can be applied even to feature spaces with a dimensionality exceeding the number of records in the dataset. New records are classified by means of the sign of the function:

y(x) =

N

X

n=1

a_nt_nκ(x, x_n) + b. (4.25)

A constrained optimization of this form satisfy the Karush − Kuhn − T ucker (KKT) conditions that in this case imply:

a_n≥ 0 n = 1, . . . , N,

tny(xn) ≥ 1

a_n{t_ny(x_n) − 1}

(4.26)

Those records from the training set resulting in an = 0 will not contribute to the prediction in Equation (4.25), the remaining records are called supportvectors, since for those points an> 0, from third Equation in (4.26) we have that t_ny(x_n) = 1, which means that they lie on the maximum margin hyperplane. Therefore in SVM models maximum margin hyperplane is defined by the support vectors, thus the solution of a classification problem is independent form the other points [13].

In most of the cases its not possible to construct a separating hyperplane without cases of misclassification, thus the target became to find an optimal hyperplane, which minimizes the classification error; for these purpose we introduce the so called slack variables ξn, where n = 1, . . . , N and ξ_n ≥ 0. When the data point is either on the margin boundary or inside it, we have ξn= 0, for all other points ξn= |tn− y(xn)|. Therefore for records on the right side of the separating hyperplane but inside the margin 0 < ξn≤ 1, for points laying on the separating

hyperplane ξn= 1, while for misclassified records ξn > 1, see Figure 4.5. With the introduction of the slack variables the costraint in (4.26) became:

t_ny(x_n) ≥ 1 − ξ_n (4.27)

Figure 4.5: Red and Green points represent the data clusters to be classified, in this case it is not possible to find an hyperplane which perfectly separate the two clusters, in this example there is a green point on the right side of the hyperplane and a red one on the left side. Slack variables ξiallow some points to fall over the separation hyperplane but penalize them.

In this way, some records of the training dataset can be misclassified, allowing overlapping between the two classes and is called sof t margin constraint. The optimization problem corre-spond then to the minimisation of the functional:

L(a) =˜ PN

The dual Lagrangian takes exactly the same form of the separable case, but with different constraints. Predictors for the test records take the same form of Equation 4.25, that is:

y(x) =

N

X

n=1

antnκ(x, xn) + b. (4.30)

Predicting future haemoglobin fluctuation is clearly a regression problem and SVM works also as regressor preserving the same properties discusses so far. First of all, with respect to the standard error function, to obtain a sparse solution the  − intensive error f unction of the form:

where  > 0 is introduced [112]. In this way whenever the difference between the actual target and the model output is lower than  prediction error is considered equal to 0.

For each data point in the training set two slack variables ξ_n ≥ 0 and ξ_n⁰ ≥ 0 have to be introduced in order to permit the prediction to be outside the hypersphere centred in tn and of radius :

y(xn) ≥ tn−  − ξn

y(x_n) ≤ t_n+  + ξ_n⁰

(4.32)

thus the SVM regression error function take the form

CX

n = 1N (ξ_n+ ξ_n) +1

2kwk² (4.33)

subject to the constrain in 4.32 and ξn≥ 0, ξn ≥ 0. As for the classification problem the mini-mization of the regression error in 4.33 can be obtained introducing the Lagrangian multipliers and solving the dual problem [13].