Once we map the target to the input clusters, a color transfer is performed between each pair of corresponding clusters. Our color transfer consists of a color transformation and a local chromatic adaptation. The color transformation is handled by the function PerformTranspOnAB(f(sI)
lab ,g
(sJ)
lab )in algorithm1. We carry out the color transformation in
the CIE Lab color space. We separate the luminance channel from the chroma channels. Indeed, the human eye is much more sensitive to changes in the light conditions than to changes in the colors. Therefore, we apply the color transformation only on the chroma channels (whatever the determined policy). Finally, we use local chromatic adaptation transform (CAT), handled by the function CATLocal(Orgb, Jrgb)in algorithm1, to repro-
duce the lighting conditions of the target image. These steps are discussed in the following subsections.
3.4.1 Color transformation on the chroma channels
The clustering step of our method performs a partitioning of the input and target images into homogeneous clusters in the Lab color space. Their 2D ab distributions can be mod- eled by 2D Gaussian distributions. Therefore, a parametric color transfer approach is used for carrying out the color transfer between the cluster ab distributions.
We adopted the parametric color transfer method, proposed by Pitié et al. [60]. Pitié’s method builds a mapping t(I) between the image input I and the target image J. The
mappingt(I) transforms the input distribution f (I) into a distribution similar to the target
distributiong(J). To build the mapping t(I), Pitié et al. assume that f (I) and g(J) follow
a multivariate Gaussian law. For each corresponding pair of input and target Gaussian clusters I(k)laband J(k)lab, we build a mappingtk(I(k)lab) consistent with the proposed mapping by
Pitié et al. [60], i.e. derived as a closed-form solution [60–62] to an optimal transportation problem well-known as the Monge-Kantorovich optimization problem [63]. That way, for each pair of clusters, we build a unique mapping which minimizes the overall cost of the color transfer [60]. More details and discussion about the Monge-Kantorovich transformation are presented in chapter6.
3.4.2 Overlapping
When using a clustering technique, we need to take care of the strong color difference which may occur between the clusters. To achieve a smooth transition between the clus- ters and to lessen the visibility of eventual artifacts caused by the color transformation, we let the input clusters overlap around their spatial boundaries. Associating pixels with more than one cluster is known as fuzzy (soft) clustering [64,65]. Each pixel i is assigned
of the 3D distributionsf(sI)
lab andg
(sJ)
a probabilitypikto belong to a cluster k as follows: pik = αik/ N X j=1 αij (5.3)
where N is the number of clusters and PN
j=1pij = 1. Additionally, αik
is defined asαik =
exp(−D2
M(xi, fCS(k))), where DM(xi, fCS(k)) is the Mahalanobis distance [66] for a 3D vector
xiwith values in the Lab color space for theithpixel. The Mahalanobis distance measures
the distance of each overlapping pixel to one of the Gaussian cluster distributions in our model. Finally, the values of the a and b channels for the output image Olabare computed
as follows: Oi = N X j=1 pijtij (5.4)
where tij is the vector of chroma values for theithpixel, obtained from the transformation
for the jth cluster. The output O
i is the vector of the transformed chroma values for the
ith pixel. Results, obtained after the color transformation in our method, as shown in
figures5.8,5.9and5.13.
3.4.3 Local chromatic adaptation
The color transformation, described in the previous section, is applied only on the chroma channels of the input image. Therefore, to complete the color transfer, we need to repro- duce the light of the target image. To perform this task, Bonneel et al. [6] apply a naive histogram matching on the luminance channel of CIE Lab. The naive histogram matching for the luminance channel may cause artifacts and highly saturated results for images with very different lighting set-ups. This makes the naive histogram matching unsuitable for our purposes, as shown in figure5.8(result (b)).
As a final stage of our color transfer method, we apply a local CAT algorithm on the image Orgb, obtained with the Monge-Kantorovich color transformation. Local CAT
aims to adapt pixel-wise the colors of image Orgb to the target illuminant. This way,
undesired color saturation is avoided and naturalism is preserved. Similarly to the iCAM algorithm [21], we apply CAT locally to the pixels of the input image by computing a “white image" using a Gaussian low-pass filter with a kernel size equal to half the sum of the image dimensions. Each input pixel is influenced by the chromatic transform and therefore, local luminance variations are captured efficiently and reproduced in the result. Indeed, the local CAT enhances the contrast and prevents the image from becoming flat (refer to examples 3 and 5 in Figure5.9).
Frigo et al. [20] were the first to apply the CAT algorithm iteratively. Instead of adapt- ing the colors of an image to a well-known illuminant, they have used a global estimation of the input illuminant and the target illuminant by assuming Gray World [18]. Likewise, we estimate the target illuminant as the average of the non-gray pixels in the target im- age [18,20]. Unlike [20], we apply the local CAT only once and not in an iterative manner, which results in a decrease of the computational time. As shown in Figure 5.8, our lo- cal CAT adapts the colors of image Orgb to the target illuminant better than the global
4. RESULTS AND EVALUATION
image. Figure5.8also illustrates the impact of the local CAT on the final result. The opti- mal color transformation of our method changes only the colors of the input image so that they become similar to the target colors. However, as we exclude the luminance channel from the transformation, the luminance of image Orgbremains relatively unchanged. As
shown in figure 5.8, once we locally adjust the colors of image Orgb to the target illu-
minant, we improve the similarity between our result and the target color palette. The influence of the local CAT on the colors of the final result is also shown in figures 5.9
and5.13.
For all results, shown in this chapter, the local CAT has been applied in the LMS color space (similarly to the CAT algorithm, presented in part2). The adaptation factor in the local CAT was scaled by 0.3 [21] and the value of the surround factor was set to 1.
Input (a) Color transfer on a and b
(c) Global CAT (d) Local CAT Target (b) Histogram
matching on L
Figure 5.8 – Comparison between the naive histogram matching, the global CAT and the local CAT. Result (a) is obtained after the color transformation for the chroma channels
a and b in our method. Result (b) is obtained by applying a histogram matching on the
luminance channel of result (a). Despite the color similarity between result (b) and the target image, result (b) is highly saturated and unnatural. Result (c) is obtained iteratively by applying a global CAT [20] on image Orgb, whereas result (d) is obtained using only one
iteration of our local CAT. This figure also presents the impact of the local CAT on the final result. Our color transformation modifies only the chroma channels of CIE Lab and keeps the input luminance relatively unchanged, as shown in result (a). The local CAT further improves the similarity between result (d) and the target color palette (note the background color and the better illuminated flower petals in result (d)).