Aplicación de Métodos Avanzados de Previsión.

GPUs are typically programmed using a Single Program Multiple Data (SPMD) programming model, such as NVIDIA CUDA [74] or OpenCL [56]. Unless stated otherwise, the CUDA naming conventions are used in the rest of the thesis.

2.3.1 C for CUDA

Although the programming framework is called C for CUDA, we refer to it simply as CUDA in the rest of the thesis. CUDA [74] is divided into two parts:

1. A C/C++ API that exposes GPUs to programs and allows them to manage their memory and launch computations to be run on the GPUs.

1_{In systems with multiple CPU sockets, the inter-CPU interconnect (e.g., HyperTrans-}

port/QPI) might need to be traversed, too. Unfortunately, current systems do not support routing remote GPU↔GPU requests over the inter-CPU interconnect. Therefore, all the experiments presented in this thesis that require the remote memory access capability are performed on systems with a single CPU socket.

2.3. PROGRAMMING MODELS FOR GPU-BASED SYSTEMS

2. A C++-like programming language to write computations that run on the GPU.

CPU-GPU interface

Programmers need to manage GPU and host memories. Although they are presented with a UVAS, they control in which GPU memory is physically allocated. They need to copy the necessary data to the GPU memory be- fore launching any GPU computation that uses it. Conversely, output data computed on a GPU needs to be copied back to host memory before it can be accessed by the CPU. Thanks to the UVAS, pointers to data allocated on one GPU can be passed to a dierent GPU, since it can be transparently accessed through the PCIe interconnect (although with much higher latency and much lower bandwidth).

The GPU is a passive device that executes asynchronous commands pushed by the host code: mainly kernel launches and memory transfers. The GPU address space is abstracted with a CUDA context. By default, a single CUDA context is assigned to each GPU, but additional contexts can be created. GPUs provide several command queues which are abstracted as CUDA streams. Commands pushed to a stream are executed in order, but commands from dierent streams can execute in any order and, if there are enough available resources, concurrently. Therefore, several streams must be used to overlap computation and data transfers. CUDA provides dierent levels of synchronization: device, stream, and individual operation (using CUDA events). Event barrier operations can be also pushed to streams to guarantee inter-stream command ordering. All these abstractions need to be used in most cases in order to achieve high performance, thus greatly increasing coding complexity.

GPU programming model

Programmers encapsulate computations in functions (called kernels) that are executed in parallel by a potentially large number of threads, although each thread might take a completely dierent control ow path within the kernel. All these threads are organized into a computation grid of groups of threads (i.e., thread blocks). Each thread block has an identier and each thread has an identier within the thread block, that can be used by programmers to map the computation to the data structures. CUDA provides a weak consistency model: memory updates performed by a thread block might not be perceived by other thread blocks, except for atomic and memory fence (GPU-wide and system-wide) instructions.

CHAPTER 2. REFERENCE HARDWARE AND SOFTWARE ENVIRONMENT

Each thread block is scheduled to run on an SM, and threads within a thread block are issued in xed-length groups (the previously described warps). Each thread has its own set of private registers and threads within the same thread block can communicate through a shared user-managed scratchpad and using synchronization instructions. The number of thread blocks that can execute concurrently on the same SM (i.e., occupancy) de- pends on the number of threads and other resources needed by each block (i.e., scratchpad memory and registers). Hence, the utilization of these re- sources must be carefully managed to achieve full utilization of the GPU.

2.3.2 OpenCL

OpenCL [56] is an open programming framework maintained by the Khronos standardization organization. It shares many design principles with CUDA but its scope is far more ambitious since it targets virtually any type of computing device. However, its genericity makes it more cumbersome than CUDA to use in applications. Instead, OpenCL is more commonly used to create higher-level development frameworks or in commercial applications that need to run on a wide variety of hardware platforms.

Although being more generic than CUDA, it provides higher abstractions in some areas like memory management. For example, memory is allocated in a context instead of a device. A context can be bound to several devices and it is the OpenCL runtime who decides in which physical memory it is allocated. Due to the dierent capabilities of the devices that support OpenCL, mechanisms like remote memory accesses are not supported either. Since the proposed techniques heavily rely on these mechanisms, we use CUDA in the rest of the thesis.

Chapter 3

State of the Art

Programmability of multi-GPU systems can be tackled at dierent levels, ranging from system support to memory management and transparent com- putation decomposition and distribution.

3.1 Automatic CPU/GPU memory coherence

GPUs use dierent memories or separate memory subsystems than CPUs. This provides a performance advantage but, on the other hand, programmers are forced to use several copies of data and keep them coherent. Some solu- tions have been proposed to automatize the CPU/GPU memory coherence.

The ADSM model proposed by Gelado et al. [45] presents a Unied Vir- tual Address Space (UVAS) which is shared by CPU and GPU. ADSM allows programmers to declare objects once and use them both in GPU and CPU functions with no need for explicit memory transfers at all. The ADSM runtime transparently creates the needed copies of the objects and makes sure that they are coherent at consistency points. It uses acquire/release consistency at kernel call boundaries. ADSM is implemented in the GMAC user-level library [10] that provides eager GPU memory update to transpar- ently overlap CPU computation (e.g., data initialization) and data transfers. GMAC uses the memory protection mechanism of modern CPUs to detect with pages are accessed in the host code. However the work in these thesis targets multi-GPU systems while ADSM is restricted to a single GPU. In Chapter 5, we present the HPE model that extends ADSM to multi-GPU systems.

While ADSM greatly simplies GPU development, programmers still need to use a specialized allocation/free functions for data that is shared between CPU and GPU. Jablin et al. propose compiler analysis in [52] to