Enfoque territorial

There are two categories of graph data systems: graph databases and graph computing engines. • Graph databases are the ones presented until now, i.e. technologies used primarily for transactional online graph persistence and typically accessed directly in real time from an application. They are the equivalent of “normal” online transactional processing (OLTP) databases in the relational world.

• Graph computing engines (or graph analytics engines) are technologies used primarily for offline graph analytics, typically performed as a series of batch steps. They can be thought of as being in the same category as other technologies for analysis of data in bulk, such as data mining and online analytical processing (OLAP). They thus typically works on a global graph level and take advantage of multiple machines for the batch works to be executed [7,48].

Graph computing engines are designed to do things like the identification of clusters on data; or to answer questions such as: “how many relationships, on average, does anyone in a social network have?”.

Whereas some graph analytics engines include a graph storage layer, others (and arguably most) concern themselves strictly with processing data that is fed in from an external source,

and then returning the results for storage elsewhere [48].

Figure 2.17: A high-level view of a typical graph analytics engine deployment [48].

Figure 2.17 shows a common architecture for deploying a graph analytics engine. The architec- ture includes a system of record (SOR) database with OLTP properties (such as some relational or document or graph DBMSs), which services requests and responds to queries from the ap- plication (and ultimately the users) at runtime. Periodically, an ETL (Extract Transform and Load) job moves data from the system of record database into the graph analytics engine for offline querying and analysis [48].

We now give the main reasons why graph databases are not the right instrument for graph analytics operations. The processing of graphs generally entails mostly random data access. For large graphs that cannot be stored in memory, random disk access becomes a performance bottleneck. However, even in the case of smaller graphs that could be entirely kept in main memory, since graph databases are centralized systems that have to assure ACID properties, they automatically lack the computational power of a distributed and parallel system. So when some computations have to be executed on the entire graph in a single-thread approach, it immediately emerges that computation times become prohibitive.

One option for realizing graph analytics with graph databases would be the realization of batch processes that perform multiple queries whit graph-local scope on the database; and later per- form some computations on the intermediate results in order to obtain the graph global result. However, when distributed computation is required, there already exist some products that manage such operations.

The first and most known framework that allows to do this is Apache Hadoop, an open source distributed-processing framework for large data sets that includes a MapReduce imple- mentation. With Hadoop and MapReduce, commodity computer clusters can be programmed

to perform large-scale data processing in a single pass [115].

In the MapReduce programming model - introduced by Google in 2004 - the Map func- tion takes key/value pairs as input and produces a set of intermediate key/value pairs. The framework groups all intermediate values that are associated with the same intermediate key and passes them to the Reduce function. The Reduce function receives an intermediate key with its set of values and merges them together. Unlike graph databases, MapReduce is not designed to support online query processing. MapReduce is optimized for analytics on large

data volumes partitioned over hundreds of machines [115,52].

However, Hadoop was thought for working with a big amount of files organized on the Hadoop Distributed File System (HDFS); and it was not designed mainly to support scalable process- ing above graph-structured data.

The main limitation in the MapReduce cluster computing paradigm is that it forces a par- ticular linear dataflow structure on distributed programs: it reads input data from disk; maps a function across the data; reduces the results of the map; and stores reduction results on disk. This massive use of disks for storing the elaboration results becomes a bottleneck when iterative algorithms are deployed [152].

In 2010, Google introduced the Pregel system as a scalable platform for implementing graph al- gorithms. In particular, graphs are inherently recursive data structures as properties of vertices depend on properties of their neighbors which in turn depend on properties of their neighbors. As a consequence many important graph algorithms iteratively recompute the properties of each vertex until a fixed-point condition is reached. The Pregel system targeted such problematics [120].

2.5. Graph database characteristics 31

GraphX is a distributed graph processing framework on top of Apache Spark. It provides two separate APIs for implementation of massively parallel algorithms (such as PageRank): a Pregel abstraction, and a more general MapReduce style API. GraphX has full support for

property graphs and it is part of Apache Spark [152].

Apache Spark provides programmers with an application programming interface centered on a data structure called the resilient distributed dataset (RDD), a read-only multiset of data items distributed over a cluster of machines, that is maintained in a fault-tolerant way. In particular, Spark’s RDDs function as a working set for distributed programs that offers a

(deliberately) restricted form of distributed shared memory [152].

The availability of RDDs facilitates the implementation of both iterative algorithms, that visit their dataset multiple times in a loop, and interactive/exploratory data analysis, i.e., the repeated database-style querying of data. The latency of such applications (compared to Apache Hadoop, a popular MapReduce implementation) may be reduced by several orders of magnitude [152].

In this work we will only analyze some of the products belonging to the former category: graph databases.