CAPITULO 1: LO ESTILÍSTICO, EL LÉXICO Y LA CRÓNICA PERIODÍSTICA:
1.4 El sistema léxico-semántico
1.4.4 Relaciones semánticas externas
Let output video file size be fsand frame resolution beRs= f ramewidth×f rameheight and the number of frames beN.
∴forone-streamalgorithms, the bpp is calculated as:
bpp= ( fs1
N×Rs)×8 (6.1)
Similarly, for thetwo-streamalgorithms, the total bpp is calculated as:
bpp= (f s1+f s2
N×Rs )×8 (6.2)
the output bitrate is determined as:
bitrate=bpp×Rs×f rames/sec(f ps) (6.3)
6.4
Objective evaluation
This section demonstrates the results obtained from the objective evaluation following the methodology described in Section 6.3. First, it introduces the coding errors produced by each algorithm followed by a detailed RD performance evaluation of the six algorithms against a set of 39 sequences and finally followed by the RD characteristics of the six algorithms for the short-listed set of six sequences.
6.4.1 Coding errors
Before, going into the performance evaluation of the six compression algorithms, it is im- portant to check the coding errors produced by the algorithms. The coding errors produced by each of the six algorithms can be obtained by following the methodology shown in Fig- ure 6.2, barring the usage of the video codec. Input HDR frames were converted to an encoder suitable HDRV. The HDRV is subsequently decoded using the corresponding de- coding function of each algorithm to reconstruct the HDR frames. Video codecs are not used in this work flow. This pipeline tests the maximal reproduction capability of the algo- rithms without the codec introduced distortions. Since perceptual metrics are designed to predict subjective quality assessment scores, Figure 6.3 shows the coding errors of the six algorithms for the perceptual metrics such as puPSNR and HDR-VDP (averaged across the six short-listed sequences).
(a) Coding errors - puPSNR (higher is better) (b) HDR-VDP (higher is better) Figure 6.3: Coding errors of the six compression algorithms averaged across six sequences with 95% confidence interval.
Explanation: It is expected that without the encoder introduced distortions, the recon-
struction capability of the compression algorithms should be maximal. However, Figure 6.3 shows that the reproduction capability ofbackward compatible algorithms are signifi- cantly lower compared to thenon-backwardcompatible counterparts. This anomaly can be attributed to the fact that thebackwardcompatible algorithms were designed to take advan- tage ofdual-loopencoding scheme (described earlier in Chapter 3), a facility not provided in this pipeline. This can be reaffirmed by the enhanced performance of same algorithms upon the introduction of the codec as shown in Figure 6.4.Non-backwardcompatible algo- rithms, on the other hand, have no such requirements.
6.4.2 Generalised RD characteristics
This section demonstrates the generalised RD characteristics of the six algorithms upon in- troduction of the video codec. In this pipeline, the HDRVs from the algorithms for each of the 39 sequences are encoded using the parameters mentioned in Sections 6.3.4 and 6.3.5. Subsequently, the video frames are decoded and reconstructed HDR frames are assessed by the seven full-reference QA metrics. Figure 6.4 shows the full set of results obtained from the seven QA/VQA metrics averaged across 39 sequences. For better clarity, logarithmi- cally scaled plots are used to demonstrate the results.
Although, the RD characteristics presented in Figure 6.4 demonstrate the overall performance of individual algorithms, the results plotted from raw data points do not give a complete perspective. Some of the data points especially forbackward compatible algo- rithms are of the order of≥10 bpp which is clearly impractical for storage and transmission requirements. Also, the results were plotted against a large set of HDR video sequences. Therefore, individual algorithms are expected to exhibit variation in both image quality as well as in output bitrate. Figure 6.5 shows the results obtained by fixing output bitrates and interpolating image quality variation across 39 sequences with 95% confidence interval bounds1 and Figure 6.6 shows the RD characteristics obtained by fixing quality levels and reporting the variation of output bitrates across 39 sequences.
Although Figures 6.4, 6.5 and 6.6 present a generalised set of results for each of the six algorithms, they cannot be directly used to predict the image quality of the six short- listed. Therefore, in Figure 6.7, the RD characteristics of the six short-listed sequences are shown. These results can be directly used to correlate the objective and subjective evaluation results and conduct a combined analysis as discussed later in Section 6.6.
6.4.3 Short-listed RD characteristics
Figure 6.7 shows the RD characteristics plotted from the raw data points for the six short- listed sequences used for the subjective evaluation. Results are presented in a logarithmic scale for clarity.
Next, similar to Figures 6.5 and 6.6, the interpolated set of results for fixed bitrates and fixed quality levels are presented in Figures 6.8 and 6.9, respectively.
6.4.4 Analysis
This section analyses the results obtained from the generalised RD characteristics as shown in Figures 6.4, 6.5 and 6.6 as well as the RD characteristics obtained from the short-listed sequences as shown in Figures 6.7, 6.8 and 6.9, respectively. A few salient points can be inferred from the objective evaluation results:
1For practical purposes quality variation up to 2.5 bpp is shown. Higher output bitrate (bandwidth) is rarely
Output bits/pixel (bpp) - log scale 10-3 10-2 10-1 100 101 PSNR-RGB [dB] 25 30 35 40 45 50 55 60 65
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(a) PSNR results (higher PSNR - better quality)
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 logPSNR - RGB 15 20 25 30 35 40 45 50 55
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(b) logPSNR results (higher is better)
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 puPSNR [dB] 20 25 30 35 40 45 50 55 60
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(c) puPSNR results (higher is better)
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 puSSIM 0.6 0.7 0.8 0.9 1
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(d) puSSIM results (higher is better)
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 Weber MSE 10-5 10-4 10-3 10-2 10-1 100
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(e) Weber MSE results (lower is better)
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 HDR-VDP(Q) 20 30 40 50 60 70 80 90
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(f) HDR-VDP(Q) results (higher is better)
Output bits/pixel (bpp) - log scale
10-1 100 101
HDR-VQM
10-2 10-1 100
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(g) HDR-VQM results (higher is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 PSNR-RGB [dB] 30 35 40 45 50 55 60 65 70 75
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(a) PSNR results (higher is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 logPSNR-RGB 20 25 30 35 40 45
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(b) logPSNR results (higher is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 puPSNR [dB] 30 35 40 45 50 55 60 65 70 75
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(c) puPSNR results (higher is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 puSSIM 0.8 0.85 0.9 0.95 1
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(d) puSSIM results Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 Weber MSE 0 0.02 0.04 0.06 0.08 0.1
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(e) Weber MSE results (lower is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 HDR-VDP(Q) 30 40 50 60 70 80 90 100
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(f) HDR-VDP(Q) results (higher is better)
Output bits/pixel (bpp) 0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25 2.5 2.6 HDR-VQM 0 0.2 0.4 0.6 0.8 1
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(g) HDR-VQM results (higher is better)
Figure 6.5: RD characteristics - fixed bitrates and interpolated quality levels with 95% confidence interval bounds (presented in linear scale).
PSNR-RGB [dB] 25 30 35 40 45 50 55 60
Output bits/pixel (bpp) - log scale
10-2 10-1 100 101
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(a) PSNR results (higher is better)
log PSNR-RGB 15 20 25 30 35 40 45
Output bits/pixel (bpp) - log scale
10-2 10-1 100 101
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(b) logPSNR results (higher is better)
puPSNR [dB] 20 30 40 50 60
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(c) puPSNR results (higher is better)
puSSIM 0.4 0.5 0.6 0.7 0.8 0.9 1
Output bits/pixel (bpp) - log scale
10-4 10-3 10-2 10-1 100 101 hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(d) puSSIM results (higher is better)
Weber MSE
10-3 10-2
Output bits/pixel (bpp) - log scale
10-1 100 101
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(e) Weber MSE results (lower is better)
HDR-VDP(Q) 30 40 50 60 70
Output bits/pixel (bpp) - log scale
10-2 10-1 100 101
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(f) HDR-VDP(Q) results (higher is better)
HDR-VQM 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Output bits/pixel (bpp) - log scale
10-1 100 101
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(g) HDR-VQM results (higher is better)
Output bits/pixel (bpp) - log scale 10-4 10-3 10-2 10-1 100 101 PSNR-RGB [dB] 35 40 45 50 55 60 65 70 75 80
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(a) PSNR results (higher is better)
Output bits/pixel (bpp) - log scale
10-4 10-3 10-2 10-1 100 101 logPSNR 10 15 20 25 30 35 40 45 50 55 60
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(b) logPSNR results
Output bits/pixel (bpp) - log scale
10-4 10-3 10-2 10-1 100 101 puPSNR [dB] 20 25 30 35 40 45 50 55 60 65
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(c) puPSNR results
Output bits/pixel (bpp) - log scale
10-4 10-3 10-2 10-1 100 101 puSSIM 0.5 0.6 0.7 0.8 0.9 1
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(d) puSSIM results
Output bits/pixel (bpp) - log scale
10-4 10-3 10-2 10-1 100 101 Weber MSE 10-5 10-4 10-3 10-2 10-1 100
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(e) Weber MSE results
Output bits/pixel (bpp) - log scale
10-4 10-3 10-2 10-1 100 101 HDR-VDP(Q) 20 30 40 50 60 70 80
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(f) HDR-VDP(Q) results
Output bits/pixel (bpp) - log scale 10-2 10-1 100 101
HDR-VQM
10-2 10-1 100
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(g) HDR-VQM results (higher is better)
Figure 6.7:Averaged RD characteristics (quality vs output bitrate) of six HDR video compression algorithms against seven QA metrics over six short-listed sequences.
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 PSNR-RGB [dB] 35 40 45 50 55 60 65 70 75
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(a) PSNR results (higher is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 logPSNR-RGB 20 25 30 35 40 45 50 55
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(b) logPSNR results (higher is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 puPSNR [dB] 35 40 45 50 55 60 65
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(c) puPSNR results (higher is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 puSSIM 0.92 0.94 0.96 0.98 1
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(d) puSSIM results Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 Weber MSE 10-3 10-2
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(e) Weber MSE results (lower is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 HDR-VDP (Q) 30 40 50 60 70 80
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(f) HDR-VDP(Q) results (higher is better)
Output bits/pixel (bpp) 0 0.5 1 1.5 2 2.5 HDR-VQM 0.4 0.5 0.6 0.7 0.8 0.9 1
fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(g) HDR-VQM results (higher is better)
PSNR-RGB [dB] 25 30 35 40 45 50 55 60
Output bits/pixel (bpp) - log scale
10-4 10-3 10-2 10-1 100 101 hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(a) PSNR results (higher is better)
logPSNR-RGB 20 25 30 35 40 45
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(b) logPSNR results (higher is better)
puPSNR [dB] 20 25 30 35 40 45 50 55 60 65
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(c) puPSNR results (higher is better)
puSSIM 0.5 0.6 0.7 0.8 0.9 1 1.1
Output bits/pixel (bpp) - log scale
10-4 10-3 10-2 10-1 100 101 hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(d) puSSIM results (higher is better)
Weber MSE
10-3 10-2 10-1
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(e) Weber MSE results (lower is better)
HDR-VDP 30 40 50 60 70
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100
hdrv hdrmpeg hdrjpeg rate gohdr fraunhofer
(f) HDR-VDP(Q) results (higher is better)
HDR-VQM 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Output bits/pixel (bpp) - log scale
10-3 10-2 10-1 100 101 fraunhofer gohdr hdrjpeg hdrmpeg hdrv rate
(g) HDR-VQM results (higher is better)
Figure 6.9:Averaged RD characteristics (quality vs output bitrate) of six HDR video compression algorithms against seven QA metrics over six sequences. Results presented in log scale.
1. The RD characteristics of the algorithms exhibited against perceptual QA metrics such aspuPSNR,HDR-VDPandHDR-VQM, as shown in Figures 6.4 and 6.7 demon- strate that non-backward compatible algorithms using higher bit-depth outperform theirbackwardcompatible counterparts at lower output bitrate.
2. Amongst thenon-backwardcompatible algorithms, thepuPSNR, puSSIM, HDR-VDP
and HDR-VQMresults exhibit thathdrvexhibits a superior HDR image reconstruc-
tion performance thanfraunhofer and lower output bitrates. This is ratified both by Figures 6.5 and 6.8.
3. The inherent design of the backward compatible algorithms require a much higher bitrate to reproduce an acceptable image quality (without H.264 blocking artefacts). The mean output bitrate for hdrjpegand gohdr (backward compatible algorithms - with residual streams containing the luma channel only) are similar to each other. The exceptions are hdrmpeg and rate. In hdrmpeg, both the base and residual streams contain 3-channels and are encoded withHigh 4:4:4 sub-sampling. Again in rate, the Lagrangian optimization applied to the residual stream reduces the overall output bitrate, albeit at the cost of image quality.
6.5
Subjective evaluation
Most full reference QA metrics, were designed to evaluate image pairs without taking psy- chophysical aspects of the human visual system into consideration. Although perceptual QA metrics are good indicators of perceived image quality, the variation in objective results emphasizes the requirement for a comprehensive subjective evaluation.
6.5.1 Design
Multiple subjective evaluations at different image quality levels are ideally required to ver- ify and correlate the results with objective evaluation. However, such an undertaking is very time consuming. Therefore, this work presents the results of two ranking-based psy- chophysical evaluations at two different quality levels. A ranking-based evaluation was chosen since it requires only one HDR display and guarantees that each ranked compres- sion technique has a unique value, thereby ensuring quick and decisive results as opposed to a full-pairwise comparison experiment. Also, the relative rapidity of the process, approx- imately 20 minutes per participant, reduces fatigue.
The primary goal of the experiments was to rank and identify the order of each algorithm, across the six short-listed sequences, at two different quality levels. Participants were tasked to rank six algorithms for each of the six sequences, one at a time. For each sequence they had to view HDRVs from each algorithm at least once. They were tasked to identify and rank the given HDRVs in order of their resemblance to the clearly labelled
reference HDRV. Also ahidden reference, identical to the labelled reference was mixed with the algorithms.
The sequences and algorithms were randomly presented in order to avoid bias. While ranking the sequences, participants were allowed to view the HDRVs as many times as required. The motivation behind this was to be able to distinguish between HDRVs that are relatively close in quality without the exhaustive full-pairwise comparisons.
6.5.2 Materials
Software resources included HDRVs from six compression algorithms, uncompressed ref- erence HDRVs and a graphical user interface (GUI) for the ranking-based experiment. Hardware resources included a SIM2 HDR display [SIMa] with a peak luminance rating of 4000cd/m2, an LG 22′′ LED display with peak luminance rating of 300cd/m2 and a
computer with a solid state drive for quick loading of HDRVs. HDRVs for psychophysical experiment
Two fixed bpp(s) representing two quality levels were selected based on the objective results shown in Figures 6.7 and 6.8, respectively. The lower quality (LQ) level was chosen at 0.15 bpp (≈8.8 Mbps - similar to online streaming quality), such that the image-quality distortions are clearly visible but not obscured by H.264 blocking artefacts. The higher quality (HQ) level was chosen at 0.75 bpp (≈44.49 Mbps - similar to blu-ray quality).
Following the chosen quality levels, the six sequences were encoded at different QP settings for each algorithm to achieve the closest possible match to the target bitrate (within 5% error margin). Subsequently, the reconstructed HDR frames were converted to a custom file format suitable for displaying the HDR frames at 30 fps on a SIM2 HDR display. Table 6.2 demonstrates the target versus the achieved bitrate for each of the six algorithms along with the error margin.
Software for psychophysical experiment
A custom GUI application, shown in Figure 6.10, was specifically built for the ranking- based subjective evaluation. It presents seven thumbnails each linked to an HDRV (labelled A-G), six from different algorithms and a hidden reference for each sequence, on the left side of the screen. The clearly marked reference HDRV (or ground truth) thumbnail is presented in the centre. Each thumbnail, whendouble-clicked plays the linked HDRV on the HDR screen. Participants are tasked to view the reference HDRV first and subsequently rank the HDRVs on the left side in order of resemblance with the reference by dragging their preferred choice to its corresponding position (labelled 1-7) on the right side. The instructions for carrying out the experiment is clearly described in a text box below the reference thumbnail.
Algorithm Target bpp Achieved bpp Error hdrv 0.15 0.148 1.33% hdrmpeg 0.15 0.159 6.00% hdrjpeg 0.15 0.155 3.30% rate 0.15 0.157 4.66% gohdr 0.15 0.157 4.66% fraunhofer 0.15 0.161 7.33% Average 0.15 0.156 4.00%
(a) Target vs achieved bpp for the LQ experiment
Algorithm Target bpp Achieved bpp Error
hdrv 0.75 0.71 5.30% hdrmpeg 0.75 0.72 2.60% hdrjpeg 0.75 0.76 1.33% rate 0.75 0.77 2.66% gohdr 0.75 0.76 1.33% fraunhofer 0.75 0.76 1.33% Average 0.75 0.74 1.33%
(b) Target vs achieved bpp for the HQ experiment
Table 6.2: Target vs achieved output bpp with error margin for lower and higher quality HDRVs
Figure 6.10:Screenshot of the evaluation software
6.5.3 Participants
A total of 64 participants were divided into two groups, 32 for each experiment (LQ and HQ), with an age range of 20 to 50 years and from various academic and corporate back- grounds took part in the experiments. The participants reported normal or corrected to normal vision.
Figure 6.11: Psychophysical experiment setup.
6.5.4 Environment
Following ITU-R recommendations [ITU12], the experiments were conducted in a room with minimal ambient lighting (below 25 lux) which is within the recommended luminance levels for a typical dark environment [Eng]. The distance between the HDR display and the participant was set to approximately 3.2 times the height of the HDR display; at a distance of≈189 cm with an LCD monitor placed at an angle of 45◦(see Figure 6.11). In order to
minimize glaring, the brightness and contrast of the LCD monitor was reduced to 25%.
6.5.5 Procedure
The participants were introduced to the objectives of the experiment prior to the start fol- lowed by a brief training session using a particular sequence subsequently discarded from the results. Upon completion of the training, the participants were asked to proceed further and rank the decoded HDRVs for the six sequences.
Each participant had to first view the reference HDRV on the HDR screen. Subse- quently, the participant had to view each of the seven decoded HDRVs including the hidden reference and perform a qualitative assessment as to how much the decoded HDRVs re- sembled the ground truth HDRV in the centre. Based on their judgement, the participants positioned the corresponding thumbnails to one of the blank positions on the right, labelled [1-7], 1 being an HDRV with least distortion compared to the reference and 7, being the HDRV with most visible distortions.
6.5.6 Results
This section provides an overview of the results obtained from the psychophysical experi- ments and analyses the same.
Let the Null HypothesisH0be that there are no significant differences between the
compression algorithms for both LQ and HQ. The alternativeH1 states that there are sig-
nificant differences between the algorithms. The statistical significance pis assumed to be 0.05. The sample size for both LQ and HQ is 32. Also, ifH1is true, it is important to deter-
mine the coefficient of concordance which measures the degree by which the participants mutually agree on choices.
Now, let A(N, M, S) be a 3-dimensional data array where N denotes all participants,