Simulaciones para la carga tipo 2

The most widespread class of parallel machines are distributed systems. A

distributed system consists of multiple processing units where each unit has its own private memory. Data is partitioned over the separate nodes and explicit communication (e.g. message passing) is required to synchronize computation. Scaling out refers to adding more processing units to the system [100]. With cloud computing this type of scaling is available through Infrastructure as a Service (IaaS) [4].

Recently, many MapReduce [31] based general purpose data processing frame- work such as [17] [10] [141], and graph processing frameworks such as Pegasus [62] and GPS [113], started exploiting efficient parallel processing of large vol- umes of data on the distributed systems. In this section, we review some well known distributed graph processing systems proposed in the literature.

Pregel [93] is a distributed graph processing system developed by Google. The design of its vertex centric model is based on BSP (bulk synchronous

2. Literature Review

Ac#ve Inac#ve

Volt to halt

Message Received

Figure 2.1: The Vertex State Machine in Pregel

parallel)[131]. Pregel partitions a graph based on vertices. Each graph par- tition contains a set of vertices and the edges associated with these vertices. Each partition is processed by one compute node in the system. In Pregel, the user defines the computation function. And this computation function will be invoked iteratively until the final result is found.

The iterative process in Pregel is performed as follows: in each iteration (or a superstep in Pregel’s term), each active vertex (the vertex that needs to be computed) invokes the computation function. In the computation function, the vertex first reads the message sent from by its neighbours in the last iteration, computes the updated value based on the message received, and then sends the updated value to its neighbours through its outgoing edges. Its neighbours will receive this value as a message in the next iteration. Once a vertex sends the message, its state will change to inactive. An inactive vertex will become active when it receives a new message. The whole process terminates when there is no active vertex in the system. This process is called Vertex State Machine in Pregel and is illustrated in Figure 2.1.

Pregel uses message passing to communicate between different compute nodes. In each iteration, the number of messages send from one vertex to its neighbours is unlimited. The corresponding receiver will read these messages in the next iteration. In the distributed environment, in order to reduce the communication cost between compute nodes, the user can define a combination function to group multiple messages into one message and send it to another machine.

2. Literature Review

to process the graphs with the scale of trillions of edges [23]. Facebook added the multi-threading support to the Giraph. So the performance of graph loading, writing and computation have been improved compare to Giraph and Pregel.

GraphLab [86] [85] is an asynchronous distributed shared-memory system. Unlike the synchronous communication in the BSP model, the GraphLab devel- ops an asynchronous model called GAS (gather, apply, scatter) to implement its parallel computation in the distribute environment. A program in GraphLab implements a user-defined GAS function for each vertex. To avoid the imbal- anced workload caused by high-degree vertices in graphs, a recent version of GraphLab, called PowerGraph [43], introduces a new graph partition scheme to handle this challenge. PowerGraph uses an algorithm called Vertex-cut parti- tioning, which partitions an input graph by cutting the vertex set, so that the edges of a high-degree vertex will be handled by multiple compute units. As a trade-off, vertices are replicated across compute units, and communication among compute units are required to guarantee that the vertex value on each replica remains consistent.

The GAS model works in the following way: in the Gather phase, each active vertex collects information from its neighbours’ vertices and edges. In the Apply phase, each active vertex can update its value based on the infor- mation gathered and its old value. Finally, in the Scatter phase, each active vertex can activate the adjacent vertices. However, unlike Pregels message pass- ing paradigm, GraphLab can only gather information from adjacent edges and scatter information to them, which limits the functionality of the GAS model.

GraphLab maintains a global scheduler. The compute units fetch vertices from the scheduler for processing, and add the neighbours of these vertices into the scheduler if needed. The GraphLab engine executes the user-defined GAS function on each active vertex until no vertex remains in the scheduler. The GraphLab scheduler determines the order to activate vertices, which enables GraphLab to provide both synchronous and asynchronous scheduling.

2. Literature Review Update1(v1) Update2(v5) Update1(v3) Data Data Data Data Data Data Data Data Data Data Data Data Data 1 Data 2 3 4 Update1(v9) CPU1 CPU2 CPU3 Execute Update

Shared Data Table Data Dependency Graph

Figure 2.2: The execution model of Graphlab

to each vertex and edge during the Apply phase is committed immediately and visible to subsequent computation. Asynchronous execution can accelerate the convergence of some algorithms. For example, the PageRank algorithm can converge much faster with the asynchronous execution. However, asynchronous execution may incur the extra cost due to locking/unlocking.

GraphLab also provides synchronous execution model. In each iteration, it executes the GAS phases in order. The GAS function of each active vertex runs synchronously with a barrier at the end of each iteration. Changes made to the vertex value are committed at the end of each iteration. Vertices activated in an iteration are executed in the subsequent iteration. The whole process is shown in Figure 2.2.

GraphX [44] is a parallel graph processing system. It supports GraphLab and Pregel abstractions. GraphX is built on top of the more general data processing system Spark [141]. It introduces RDG (resilient distributed graph), which is an extension to the RDD (resilient distributed dataset) in Spark. With such extension, many graph algorithms can be implemented easily with the Spark’s

2. Literature Review

build in operation, GraphX also redesigned the vertex-cut algorithm.

Chaos [110] is a graph processing system designed for analytics on big graphs using clusters. Chaos builds on the X-Stream (discussed in detail in next sec- tion) [109] single-machine graph processing system, but scales out to multiple machines. Chaos adopts an edge-centric and GAS (Gather-Apply-Scatter) pro- gramming model. Due to the use of the edge-centric model, Chaos partitions the graph for sequential storage access, rather than for locality and load bal- ance, resulting in much lower pre-processing times. In Chaos, the graph data is distributed uniformly and randomly across the cluster and does not attempt to achieve locality, based on the observation that in a small cluster network bandwidth far outstrips storage bandwidth. In addition, Chaos uses work steal- ing to allow multiple machines to work on a single partition, thereby achieving load balance at runtime. With all these features, Chaos is limited only by the aggregate bandwidth and capacity of all storage devices in the entire cluster.