With the increased complexity of the Internet infrastructure, applications, and services, IDSs are required to be able to deal with the increasing number of malicious activities, the rise in the amount of traffic as well as the growth in bandwidth speed [23]. However, the pro- cessing capability of the payload-based IDSs is assessed at a low level, such as at between 100 Mbps and 200 Mbps with commodity hardware being employed, and approximated at 1 Gbps with dedicated hardware being used [24]. Moreover, payload-based IDSs are processed in the way of per-packet inspection that relies only on header information in or- der to identify misuses in encrypted protocols [25]. Given these issues, network-based (or flow-based) approaches seem to be promising candidates for intrusion detection research. Flowsare monitored by built-in specialized accounting modules in network routers. After exporting the reports of flow events to external collectors by the modules, network-based monitoring system will analyse these flows to suspect on malicious activities. Compared to the conventional host-based approach, network traffic based monitoring approach can be used as a complement of packet inspection, which are capable to manage a considerable amount of network traffic data. The question remains in network-based monitoring whether only the flow information is enough to be useful for intrusion detection, compared with the payload-based inspection approach. Flow measurements are the information of aggregated network traffic. So, it does not deliver the detection precision. However, flow measurements give an aggregated viewpoint of the traffic data transferred between hosts over the network. Network traffic monitoring (NTM) has been playing an important role in understand- ing and characterizing users’ activities only though the flow-based information in computer networks and systems. The aims of NTM are mainly focused on the three improvements, which include the quality of service (QoS) of the network, optimization of resource us- age, and enhancement of security in computer networks [26]. Specifically, first, network conditions can be recognized by the network manager with NTM scheme. It provides the
complete details about the QoS of network, such as bandwidth, throughput, propagation de- lay, link availability, jitter, server memory, database space and etc [27]. Second, with NTM being implemented at network nodes, i.e., network gateways, such as network routers, the network traffic that is traversing the network is under online observation [28]. Thereby, the network utilization can be improved by optimizing the resource usage to avoid the network congestions. Third, malicious activities, unauthenticated service or approaches to the server will be identified by regularly monitoring the traffic flow [29]. The network convention and statistics about the traffic will be known easily which helps to troubleshoot the network. Security events will also be investigated and the entry of the user will be maintained for responsibility. Over the years, a number of methodologies have been proposed in NTMs to understand network performance and users’ behaviour to monitor and analyse network traffic behaviours [30]. Depending on the locations where it applied in computer networks, the NTM techniques can be categorised into two main classes, that is, router-based net- work traffic monitoring(RNTM) techniques and non-router based network traffic monitor- ing (NNTM) techniques. Some RNTM methods proposed in [31–33] are hard-coded into network routers. RNTM is responsible for collecting information which is located on man- aged devices, and also execute applications that monitor and control the managed devices. On the other hand, some NNTM methods [26, 34, 35] transmit probes into the network to collect measurements between at least two endpoints in the network. It deals with metrics, such as availability, routes, packet delay, packet loss, packet inter-arrival jitter, and band- width measurements (capacity, achievable throughputs). In this research, the techniques of RNTM that built in network gateway, i.e., network routers, will be developed to monitor the network traffic characteristics against malicious activities.
1.2.1
Router-based Network Traffic Observation
Router-based Network Traffic Observation (RNTO) is network flow based monitoring mech- anism that built in network routers to capture the IP traffic information, analysing traffic characteristics and monitoring network traffic for malicious activities.
Some network traffic monitoring schemes are developed for traffic observations to im- prove the network performance. Simple network monitoring protocol (SNMP) is an applica- tion layer protocol that is part of the TCP/IP protocol suite [36]. SNMP collects the statistic information of network traffic by passive sensors at network router and destination side. By collecting and organizing information on IP networks, administrative computers have the task to monitor and manage a group of hosts in the computer networks [37]. Remote Moni- toring (RMON) enables various network monitors to exchange data of network monitoring [38, 39]. Unlike SNMP that need to send out a request for information, RMON is capable to set alarms that pre-set based on a certain criteria for network monitoring. In addition, it is allowed to manage local networks and remote sites as well. NetFlow is introduced by Cisco that provides the capability to collect IP network traffic on network routers [40, 41]. By analysing the data provided by NetFlow, the network traffic information such as the source and destination, class of service, and the causes of congestion can be determined.
However, the existing RNTO techniques in the aforementioned references analyse the traffic behaviour and characteristics in a static and statistical way. With the increased com- plexity of the network usage, the static analytical approach is incapable to online monitor the network traffic in real-time to deal with the growth in network load and attack frequency. The dynamics of network traffic in the router-based network can be represented by a fluid-flow model [42]. Moreover, the fluid-flow model can be further formulated by stochas- tic nonlinear equations to describe the traffic dynamics. With the simplified flow model describing the network traffic on routers, the control theory methods have been applied to analyse the dynamical behaviours of traffic flows on the router. These control theories-based
Router
:
Buffer
Traffic flows Forward traffic
AQM RNTO
Fig. 1.3 Router-based Network Traffic Observation (RNTO).
methods have demonstrated great effectiveness in monitoring and managing network traffic on the routers. As shown in Fig. 1.3, some active queue management (AQM) algorithms are developed to be applied into network routers to stabilize the queue length in router buffer at a desired value for congestion avoidance to manage the network [43, 44]. Besides AQM algorithms, RNTO is embedded into network routers. They have two main applications. First, it is applied in the advanced AQM algorithms for network congestion control [45– 47]. Second, it monitors the traffic flow in TCP networks [48–50]. Considering network anomaly in the router-based network as a perturbation into the network, RNTO is respon- sible to estimate and detect the network anomalies. Some methods has demonstrated great effectiveness in monitoring network traffic.