La educación en valores en los procesos educativos

Protocol Payload Connectivity Model SSH Encrypted Client-server

SMTP Plain-text Client-server POP3 Plain-text Client-server HTTP Plain-text Client-server HTTPS Encrypted Client-server Bittorrent Plain-text Peer-to-peer

Table 4.11: A list of test protocols considered in this case study, along with their payload obfuscation and connectivity model information.

each of these protocols, with specific reference to payload obfuscation and connectivity models. It is important to note that certain protocols, such as Bittorrent, operate using both plain text and encrypted communications.

In this case, each would be described by a distinct TWEANN classifier.

The application protocols chosen for accuracy comparison in this case study were those of similar works with already-published results. These application protocols, listed in Table 4.11, are common to many public networks and are thus the focus of many IP classification system tests. The specific works chosen for comparison in this case study (Table 4.8) were the most commonly cited of these works, published over the last decade.

The APIC method created a TWEANN classifier for each of the six considered application protocols listed in Table 4.11. The topology of the network, including the number of neurons and their neuronal connectivity was evolved to an optimal structure using a GA. A second GA was used to evolve the weights of each network, providing increased efficiencies compared to a local search for optimal weights from random weights adjusted by the backpropagation algorithm. Instead, the GA calculated near optimal initial weights, which the backpropagation algorithm fine-tuned, achieving convergence on a satisfactory local maximum. While 1000 training iterations were afforded to the backpropagation algorithm, in each case the algorithm achieved convergence almost immediately, as illustrated in Figure 4.34. Here, a sharp decline in the output deltas during training is observed almost immediately. This sharp decline is a testament to the efficiencies gained through initial weight selection by a GA, prior to being parsed by the backpropagation algorithm.

0 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009

0 5 10 15 20 25 30 35 40 45 50

Delta

Epochs Convergence

Min Avg Max

Figure 4.34: The convergence graph for the backpropagation algorithm executed on the topology illustrated in Figure 4.28. The graph has been zoomed to include data for the first 50 epochs only. The steep drop in delta over a short number of training epochs (less than ten) is testament to the success of APICs weight-adjusting GA.

Section 4.7 compared six similar systems to the accuracy results achieved by the APIC method. While many reported remarkably high accuracy, there were a number of caveats accompanying the published results. For example, the results published for the classifiers tested by Bernaille et al. (2006) were only achievable using their cluster and port-labelling heuristic, where an application’s “standard port” was assigned to each cluster. The authors argued that this attribute greatly improved the accuracy achieved by their system, which was evident in their results. This finding is contrary to findings in Section 4.1.1, where the use of standard ports to identify application protocols was found redundant and prone to error. While Bernaille et al. (2006) stated that port numbers were only used “when meaningful”, they agreed that the applicability of port-based classification was becoming increasingly limited due to dynamic port selection strategies.

Alshammari et al. (2009a) assert that an assumption is made, whereby all application protocols are adhering to IANA-assigned application ports.

Section 4.1.1 concluded that current trends show that an increasing number of application protocols select port numbers at random, or elect to operate on well-known ports to confuse and evade traffic shapers and firewalls. This, along with latency-sensitive metrics considered by the method, brings into question the ability of the classifiers to produce high degrees of accuracy when ported to networks other than where the initial training set was recorded.

Alshammari et al. (2009a) also recognise this in their publication, stating that tests showed that the classifiers performed poorly when deployed on other networks.

The DPI-based classifiers produced by Bujlow et al. (2015) produced exceptionally high accuracy scores for many application protocols, however all of these classifiers were created manually using regular expressions to match plain-text application protocols. This was demonstrated by Bujlow et al. (2015) attempting to classify encrypted bittorrent, where only 78.68 percent accuracy was realised. This is in accordance with the findings of Section 4.1.2, where DPI-based approaches were deemed extremely successful for identifying plain-text flows, however are rendered inept when packet payload is rendered opaque. This observation is of increasing importance, especially as the trend for application protocol developers to use encryption, prevails.

The best scoring topology for each of the six application protocols produced by APIC tracked in this experiment was illustrated in Section 4.6.

Here, the best scoring classifiers all scored in excess of 99 percent. While these results are comparable to those in Section 4.7 and Section 4.1, it is important to remember that these results were achieved autonomously, without the aid of a human expert. Where the comparable works used manual processes to annotate data sets and create classifiers, APIC was able to perform these tasks automatically using ML techniques.

Furthermore, the classifiers produced by APIC were found to be portable.

Both the automation of classifier creation and portability of these classifiers increases the completeness of IP traffic classification systems. A more complete system results in less “unknown” IP traffic flows, increasing the control administrators have over their network. In addition to being

complete, an IP traffic classification system should also be accurate. In this case study, APIC was found capable of producing classifiers that rivalled those of comparative systems, designed manually by experts over the past decade. These results are summarised in Table 4.8.