INVENTARIO HABITOS DE ESTUDIO – POZAR INSTRUCCIONES

I ran vEQ-benchmarkon a number of hardware configurations. These included two identical, high-end, hardware configurations (but with different operating systems installed), a mid-range desktop configuration and a Raspberry PI-2 device. These configurations are summarised in Table 5.2. For the workloads, I selected 10 popular videos on YouTube that are available in the 4K UHD (3096x2160 – 2160p) resolution and below. I found that YouTube makes such videos available in up to 32 different formats, identified by a 2 or 3 digit value referred to as an itag. For the purposes of this experiment, I only used itags corresponding to the following picture sizes: 320p, 480p, 720p, 1080p and 2160p, There are there are 16 different itags available for these formats. I benchmarked the first 120s of each of these videos, and repeated each measurement 5 times. Based on experience from previous work, I have found that 120s

74 CHAPTER 5. AN ENERGY BENCHMARKING TOOL FOR INTERNET VIDEO

Figure 5.2:Generated output from a singlevEQ-benchmarkbenchmarking session. It shows a section of measurements for power, CPU, memory and ingress network usage. (Other data excluded for brevity.)

of strategically captured video is sufficient to characterise the resource usage of the entire video. Additionally, a significant proportion of Internet videos (e.g. on Youtube, Instagram and Vine) are less than 200 seconds [63]. Thus, I performed a total of 800 measurement sessions of 120 secs for each of the devices benchmarked.

5.3. EXPERIMENTS 7 5 Workstation Operating System

GPU Codecs Powermeter Video player

Intel X99 chipset,

Intel i7-4920 CPU @ 4GHz, 16GB DDR4 DRAM Ubuntu Linux 14.04 Nvidia GTX 960, 2GB H.264(libx264 version r2538), VP8/VP9 (libvpx version 1.4.0.8) Voltcraft VC870 (with USB inter- face)

libVLC version 2.2.2

Intel X99 chipset,

Intel i7-4920 CPU @ 4GHz, 16GB DDR4 DRAM Windows 8.1 Nvidia GTX 960, 2GB H.264(libx264 version r2538), VP8/VP9 (libvpx version 1.4.0.8) Voltcraft VC870 (with USB inter- face)

libVLC version 2.2.2

Intel i5-4440 CPU @ 3.1 GHz, 8GB DDR4 DRAM Ubuntu Linux 14.04 None H.264(libx264 version r2538), VP8/VP9 (libvpx version 1.4.0.8) Voltcraft VC870 (with USB inter- face) libVLC version 2.2.2 Raspberry Pi 2 ARMv7 BCM2709 Quad- core CPU @ 900MHz, 1GB RAM Ubuntu Linux MATE 14.04 Videocore 4 GPU H.264 (omx- h264 version 1.1) Voltcraft VC870 (with USB inter- face)

omxplayer ver- sion

76 CHAPTER 5. AN ENERGY BENCHMARKING TOOL FOR INTERNET VIDEO

Figure 5.3: Screenshot ofvEQ-benchmarkshowing measurement of a video from [165]. The video playback has a minimal interface, with real-time measurement (in the top-right corner) that is refreshed every second. All experiments are run in windowed mode, at the size of the video resolution.

5.3.2

Results

I present results from the various experiments.

Single Run Results:

During a benchmarking session, the tool captures various measurements for the video workload. Some of these measurements, such as the instantaneous power, CPU and Memory utilisation, may be displayed as overlaid text on the video being decoded, and serves an immediate indicator of the resource usage of the particular video. This is shown in Figure 5.3. At the end of a single benchmarking run, these measurements are collated, and our tool is able to automatically generate graphical visualisations of measurements taken for that run as a time-series example(shown in Figure 5.2). This makes it possible to examine the points within the video that consumed the most, or the least, amount of resources. In this respect, our tool could be modified to facilitate very fine-grained studies of the characteristics of video files through statistical analysis or machine learning. This is an avenue for future work.

Summary Results:

As stated previously, I selected 10 videos from YouTube as workloads for the benchmarking exercise. The results presented in Figures 5.4 and 5.5 are aggregate

5.3. EXPERIMENTS 77

values of all 10 video samples for the various hardware configurations. There are no real surprising results considering the picture sizes: larger picture sizes consume more system resources - CPU, memory, network capacity and thus, more energy and. However, it is interesting to see therelativedifferences between pictures sizes (left to right across the columns in each of the figures (Fig. 5.4 and Fig. 5.5). For instance, we can see that there is a statistically significant difference between the energy usage full-HD (1080p) video and a 4K/Ultra High Definition (2160p) video of around 12W (at 95% confidence intervals). There is also the intermediate 1440p resolution between FHD and UHD. This suggests that a dynamic adaptation that switches between these resolutions over a long period of time could lead to significant energy savings. I analyse the possible impact on objective quality in Section 5.3.2.

Another interesting observation is the difference in energy usage between Ubuntu Linux 14.04 and Windows 8.1 (on identical hardware) in Figure 5.4, due to a larger idle power overhead. due to a larger idle power overhead. This was possibly due to a larger number of background processes on a Windows platform It can be observed that the Linux platform consumes less energy (about 10W) overall than Windows 8.1, suggesting that it is a more energy-efficient platform for video playback. We can also observe that the i5 platform uses less energy than both i7 workstations. This is understandable as the i5 is a lower-end CPU, and the workstation does not have an GPU installed. However, the CPU struggled with playing back 4K video. As such, there was a noticeable amount of lag observed during 4K video playback. This would have severe impacts on the user QoE.

Finally, we observe that the Raspberry Pi 2 device consumes a relatively negligible amount of energy (compared to the workstations). It consumes between 2.7W (at 480p) and 3.1W (at 1080p), with an idle power of just 2.5W. The Raspberry Pi device is equipped with a VideoCore IV BCM2763 GPU 8which is capable of playing back H.264 video at up to 1080p resolution. This low energy usage makes it a very energy- efficient device. It highlights the potential of these types of device for educational purposes and in regions where power supply is expensive or limited. This device does have a shortcoming that is common to all hardware-assisted playback in general. This shortcoming is that ofrigidity- the hardware is hard-coded to support only the H.264 codec up to 1080p resolution, which means it is impossible for it to support 4K playback or other codecs such as VP8, VP9 and HEVC.

78 CHAPTER 5. AN ENERGY BENCHMARKING TOOL FOR INTERNET VIDEO

Comparing HD, FHD and UHD formats:

In Figure 5.8, I show a more fine-grained result for playback of High Definition (720p), Full High Definition (1080p) and Ultra High Definition (2160p or 4K-UHD) resolutions on the i7 workstations. We can observe the expected trend, as the larger formats require more energy to playback. However, this difference is significant, e.g. for the ‘Samsung Galaxy S’ video9’, we observe a difference of 26J/svor 36% (HD compared to UHD) and 18J/svor 22% (HD compared to FHD).

The energy usage is grouped by the YouTubeitag value - each represents a different codec-resolution combination, showed in Table 5.3. I also show a summary of the bandwidth resource requirements of the videos aggregated by picture size (height) in Figure 5.6.

Quality Assessment I performed an objective quality assessment using the well-

known SSIM [122] metric for the ten YouTube videos used in our experiment. A representative set of this is shown in Figure 5.10. SSIM is often described as having a very good approximation to human perception. It requires that an uncompressed source video be used as a reference for comparison with a compressed, scaled source. As the source of the YouTube videos are not made available, I have made our SSIM measurementsrelativeto the best quality of video available (i.e. the 4K video at the highest bitrate – usually corresponding to an itag value of 272). This means our measured objective quality values, as well as EnVI metrics, are relative to the best quality available for that source. Making the measurements in this way is very relevant to our approach, as then the benchmark output has a reference that is the highest quality source.

I observe, as expected, that all the videos show a strong similarity to the best quality video (>0.95). This is not surprising with YouTube videos. YouTube (owned by Google) possess the technical expertise to encode optimally for video quality for a given video format. However, our SSIM measurement results indicate that most human viewers might not see a strong difference in these videos. Of course, in reality there would be several other factors and considerations to be made, which the SSIM metric does not capture. These could include screen size, distance of the viewer from the screen, brightness and ergonomics (amongst others). These human factors are often taken into consideration during standardised, subjective measurements, e.g. [132], and these factors cannot be assessed by our tool. However, I do present a subjective assessment