Marco Conceptual

This section will detail the experimental evaluation and datasets used to assess the ability of Graph Fingerprints to be used for graph comparison and classification.

3.6.1

Comparision Datasets

The synthetic graphs used throughout the comparison results section (including Forest Fire [145] and Erd˝os-R´enyi [70] random graphs) were generated using the SNAP graph analysis package [147]. The Forest Fire generation method was introduced by Leskovec, and produces more realistic synthetic graphs than the frequently used Barab´asi-Albert as it replicates more features seen in empirical graphs [145]. For all Forest Fire graphs used in the results section, the forward burning probability was set to 0.35 and the backwards burning probability set to 0.32. These values produce graphs which approximately follow |E| = |V| ∗4. The empirical

Table 3.2: Empirical graph datasets used to assess graph comparisons using Graph Fingerprints

Dataset |V| |E| %V inLCC N umT riangles

soc-Slashdot0902 82168 948464 100 602592

ca-HepPh 12008 118521 93.3 3358499

com-DBLP 317080 1049866 100 2224385

loc-Gowalla 196591 950327 100 2273138

wiki-Talk 2394385 5021410 99.8 9203519

data used was taken from the widely used SNAP datasets repository [146]. A summary of the datasets used can be seen in Table3.2. The datasets are taken from a range of domains including collaboration, communication and social networks.

Random Rewire Graph Generation

To demonstrate that the GFP-C approach is highly sensitive to changes in the underlying topology of a given graph, the edges in a Forest Fire graph with 100,000 vertices were rewired in a random manner, as detailed in Section 2.4.2. Figure 3.1shows how the degree distribution of the original graph was altered by the random rewiring process, where the number after the graph name indicates the quantity of edges rewired. The figure plots the number of vertices

N P(total) with a specified total degree valuektotal and illustrate how the internal connectivity

of the original Forest Fire graph is altered as more edges are rewired.

3.6.2

Comparision Testing Methodology and Environment

All the experiments for graph comparison presented in this chapter were performed upon a small development Hadoop cluster consisting of a head node with a 6 core Intel Xeon E5-2609v3, 64GB RAM and 1TB of SSD storage. In addition, the cluster contains 4 worker nodes each with 2 * 8 core Intel Xeon E5-2630v3, 64GB RAM and 1TB of SSD storage. All nodes in the cluster are connected via a dedicated SFP+ 10Gb network and run the same software stack of CentOS 7.2, Java 1.8, Scala 2.10.5, Apache YARN 2.7.1 and Apache Spark 1.6.1. All experiments using Spark were run using YARN to allocate cluster resources in the form of containers.

For all experiments, γ was set to 2 to increase the weightings of the global features in the final similarity score.

(a) Original Graph (b) Graph 1 (102_{) (c) Graph 2 (10}3₎

(d) Graph 3 (104) (e) Graph 4 (105) (f) Graph 5 (106)

Figure 3.1: Change In Degree Distribution After Rewiring Process.

3.6.3

Classification Dataset Generation

As outlined in Section2.4, large quantities of graph data are not readily available within the public domain. This makes the assessment of a global graph classification approach challenging as numerous (at least in the order of hundreds) examples of graphs from each of the classes being identified would need to be present, something that is not present in the standard benchmark datasets. Additionally, having data generated by a known algorithmic processes can serve as a very reliable source of ground truth. Due to these issues, two balanced synthetic datasets were created using a combination of five mathematically understood random graph generation methods from the SNAP graph library [147]. More details on the generation of random graphs can be found in Section 2.4.1. The two datasets created for the experimentation are detailed below:

Dataset One (Multi-Class) - Containing 10,000 graphs from each of the five generation

methods, creating a final dataset of 50,000 graphs, with five balanced classes. This dataset was created to test the ability of theDTC approach at multi-class classification.

Dataset Two (Binary Classification) - Containing 10,000 forest fire graphs and 10,000 randomly rewired forest fire graphs. The goal of this dataset was to test the sensitivity of the DTC approach at classifying graphs which are highly topologically similar but of two different classes.

The forest fire graphs represent a normal distribution of graphs, whereas the rewired graphs represent anomalies where small changes have been made to their topologies. The random rewire process modifies a given source graph’s topology by randomly altering the source and target of a set number of edges according to the Erd˝os-R´enyi random model. The number of edges each graph was rewired by was chosen uniformly from a possible range of 100 to 10,000.

Many of the graph generation methods used require parameters to control aspects of the generation process. To avoid our models over fitting to a particular set of generation parameters, these we uniformly randomised by the amounts detailed below. Each graph was generated with 100,000 vertices and a varying number of edges controlled via the generation method.

The chosen generation methods cover a broad range of possible graph topological structures, with a particular focus on those found in the social, web and citation domains. However the type of structures found in many biological graphs (particularly graphs of brain connectivity) are not typically covered by these generation approaches. Such graphs tend to exhibit hierarchical [190] or modular [112] structure, however we leave analysis over these types of graphs as possible future work. The final generation methods chosen were:

• Forest Fire (FF)[145] - The forward and backward burn probabilities were chosen uniformly between 0 and 0.5.

• Barab´asi-Albert (BA) [7] - The number of connections made by each new vertex joining the graph was chosen uniformly between two and six.

• Erd˝os-R´enyi [70] - No parameters were randomised for this method as edges are made at random, with each vertex having a mean degree of two.

• Small World (SW)[231] - The Watts-Strogatz Small world model was designed to generate random graphs whilst accounting for, and replicating, features seen in real-world graphs – specifically, to maintain the low average shortest path lengths of the ER model whilst increasing local clustering coefficient. The rewire probability for the small world model was chosen uniformly between 0 and 0.5.

• R-MAT (RM) [47] - The R-MAT graph generator uses a recursive matrix technique to generate realistic graphs. It requires the probability that a certain edge will fit into one of three partitions within a 2×2 matrix. These probabilities are uniformly chosen to sum to less than one, with the mean degree being two.

3.6.4

Classification Testing Methodology and Environment

All the accuracy scores presented in the results section are the mean accuracy after k-fold cross validation, considered the gold standard for model testing [11]. Fork-fold cross validation, the original dataset is partitioned into k equally sized partitions. k−1 partitions are used to train the model, with the remaining partition used for testing. The process is repeatedk times using a unique partition for testing and a mean taken to produce the final result.

Experimentation was performed on a compute system with 2 Nvidia Tesla K40c’s, 20C 2.3GHz Intel Xeon E5-2650 v3, 64GB RAM and the following software stack: CentOS 7.2, GCC 4.8.5, CUDA 7.5, CuDNN v4, TensorFlow 0.10.0, Keras 1.0.8, scikit-learn 0.17.1, Boost 1.56, Python 2.7.5 and Graph-Tool 2.8.