From Eq. (4.1), we observe that the color components at a pixel are deter-mined by three main factors, namely, reflection coefficients of the surface point whose image is formed; spectral power density (SPD) of the illuminants or ra-diations received by the scene from various sources; and the spectral responses of camera sensors. Typically, we consider the imaging of a 3-D scene under varying illumination. In that case, pixel values of images under different scene illumination vary greatly, and the colors rendered by display systems may also show these variations. However, a human observer is capable of perceiving true colors of the objects present in the scene even under this situation. Color con-stancy of a scene refers to restoration of colors from varying illumination of a scene. Its primary objective is to derive an illumination-independent rep-resentation of an image so that it could be suitably rendered with a desired illuminant. The problem has two parts. The first one attempts to estimate the spectral component of the illuminants, and in the other part, it performs color correction for rendering the image with a target illumination.
4.5.1 Estimating Spectral Components of a Single Illuminant Many techniques have been reported [12, 13, 43] for estimating the spectral components of the illuminant of a scene, with the assumption that it has a single illuminant. However, all these techniques solve the color constancy problem in the spatial representation of images in the RGB color space. For adapting them in the compressed domain, we need to perform processing in the YCbCr color space. In a simplified model [43], we assume that all the reflecting bodies have ideal 2-D flat Lambertian surfaces. According to Eq. (4.1), one of the objectives is to estimate E(λ) from an input color image, which is expressed in the form of a color vector (RE, GE, BE) in the RGB color space. It is observed [43] that the problem is equivalent to finding a solution to an under-constrained set of equations. Hence, to solve this problem, we have to consider additional assumptions or constraints imposed from the environment. These are briefly discussed below with their necessary adaptation in the YCbCr color space.
1. Gray World: In the gray world assumption [17, 50], it is assumed that the average reflectance of all surfaces is gray or achromatic. Hence, the average of color components provides the colors of the incident illumi-nant.
2. White World: In this approach [81], an attempt is made to identify a white object in the scene and the color of these white pixels in the image determines the color of the illuminant. To obtain these values, we look for the maximum values of individual color components. This method is quite sensitive to the dynamic ranges of the sensors. However, if the dynamic range of brightness distribution satisfies the linear response of the sensor, this assumption works well in many cases. This assumption is referred to here as the white world assumption. In another variation in the YCbCr color space, instead of computing the maximum values in all the three components, the computation is carried out only in the Y component. The corresponding chromatic components of the pixel with maximum Y are chosen for the color of the illuminant. This assumption is referred to as white world in YCbCr.
3. Statistical Techniques: A more recent trend in solving the color constancy problem is to use statistical estimation techniques with prior knowledge on the distribution of pixels in a color space given known camera sensors and source of illumination. In these techniques, an illuminant (or a set of illuminants) is chosen from a select set of canonical illuminants based on certain criteria. Some of these approaches are discussed below.
(a) Color Gamut Mapping: The color gamut of a scene provides the statistics of color distribution under an illumination. In this tech-nique, an illuminant is mapped to a color gamut covering all pos-sible color renditions under its exposure. There are color gamut
mapping approaches both in the 3D [44] and in the 2D [42] color spaces, where an objective is set to maximize the evidence of color maps with known maps of canonical illuminants. In a typical 2-D implementation [109], this statistics is collected in the CbCr space by discretizing it into 32 × 32 cells.
(b) Color-by-Correlation In this approach [43], the attempt is made to maximize a likelihood of an illuminant given the distribution of pix-els in the 2D chromatic space. In [43], it has also been shown that the computational framework is generic enough to accommodate many other algorithms (like the gamut mapping one in the 2D chro-matic space). In [43], the chosen chrochro-matic space is formed by nor-malized red (r = R+G+BR ) and normalized green (g = R+G+BG ) com-ponents. In the compressed domain [109], the method is adapted in the CbCr space. and statistics is collected in the same way.
(c) Nearest Neighbor Classification As the above two techniques re-quire significant amount of storage space for storing the statistics of each canonical illuminant, in [109] a simple nearest neighbor (NN) classification is used for determining the canonical illuminant. Let C ∈ Cb × Cr denote the SPD of an illuminant in the CbCr space, which follows a 2-D Gaussian distribution as given below:
p(C) = 1
2π|Σ|12e−12(C−µ)Σ−1(C−µ)t, (4.12) where µ(= [µCb µCr]) is the mean of the distribution and Σ(=
σ2Cb σCbCr
σCbCr σ2Cr
) is the covariance matrix . The canonical illu-minant is chosen based on the distance of its mean illuillu-minant from the mean chromatic component of the image. The distance func-tion used in [109] is the Mahalanobis distance funcfunc-tion [91]. Let the mean chromatic components of an image be Cm. Then, for an illuminant L with the mean µLand the covariance matrix ΣL, the distance between them is given below:
d(Cm, µL) = (Cm− µL)Σ−1L (Cm− µL)T. (4.13)
4.5.1.1 Computation in the Block DCT Space
For computing in the block DCT space, we simply estimate the color of the illuminant from the DC coefficients of individual blocks, which is the aver-age of 8 × 8 sample points in a block. However, with a marginal increase in computation, we include more number of points by taking three more addi-tional points (averages of 4 × 4 subblocks) for each block. The same strategy is followed in [113] while collecting statistics for determining color pallets in the block DCT space. This is carried out by performing an inverse 2D DCT directly over the DCT coefficients.
Table 4.3: List of algorithms for estimating the color components of an illu-minant
Algorithms Domain Short Name
Gray World Spatial GRW
Gray World Block DCT GRW-DCT
White World Spatial MXW
White World in RGB Block DCT MXW-DCT White World in YCbCr Block DCT MXW-DCT-Y Color by Correlation Spatial COR
Color by Correlation Block DCT COR-DCT
Gamut Mapping Spatial GMAP
Gamut Mapping Block DCT GMAP-DCT
Nearest Neighbor Spatial NN Nearest Neighbor Block DCT NN-DCT
Let Xij, 0 ≤ i, j ≤ 7 denote the DCT coefficients of an 8 × 8 block. From
As the computation is carried out with a lower resolution of input images, let us discuss its effect in the quality of results. As reported in [109], it is found that the technique provides comparable results in estimating the spec-tral components with those obtained from the spatial domain techniques deal-ing with the full resolution of images. To demonstrate the comparative perfor-mances of different approaches, let us consider a metric named angular error reflecting the quality of estimation of the SPD. Let the target illuminant T be expressed by the spectral component triplet in the RGB color space as (RT, GT, BT), and let the corresponding estimated illuminant be represented by E = (RE, GE, BE). Then, the angular error (∆θ) between these two is
In the above definitions, ‘.’ denotes the dot product between two vectors, and
|.| denotes the magnitude of the vector. The techniques under comparison are listed in Table 4.3.
The SPDs of the illuminants are estimated from the image data set cap-tured by Barnard et al. [14] using different illuminants. These data are avail-able at the websitehttp://www.cs.sfu.ca/∼colour/data. There are 11 illumi-nants as listed inTable 4.4. Every image in the data set has also information
Table 4.4: List of illuminants.
Illuminant Nature of Source Short Name
Philips Ultralume Fluorescent ph-ulm Sylvania cool white Fluorescent syl-cwf Sylvania warm white Fluorescent syl-wwf Sylvania 50MR16Q Incandescent syl-50mr16q
Same with blue filter syl-50mr16q+3202
Lamp at 3500K temperature Incandescent solux-3500
Same with blue filter solux-3500+3202
Lamp at 4100K temperature Incandescent solux-4100
Same with blue filter solux-4100+3202
Lamp at 4700K temperature Incandescent solux-4700
Same with blue filter solux-4700+3202
Table 4.5: Performance of different techniques of estimating illuminants Technique ∆θ (in degree)
GRW-DCT 14.77
MXW-DCT 13.82
MXW-DCT-Y 14.88
COR-DCT 6.97
GMAP-DCT 10.8
NN-DCT 7.71
GRW 14.77
MXW 29.7
COR 11.45
GMAP 11.41
NN 7.1
regarding the spectral components of the illuminant in the RGB color space.
These are used for collecting statistics for different statistical techniques. In Table, 4.5 the median of the angular errors of the estimates over the com-plete set of images is shown. We observe that compressed domain techniques performed better than spatial domain techniques in this respect.
4.5.1.2 Cost of Computation and Storage
For expressing the computational cost of different techniques, let the number of pixels in an image be denoted by n. Moreover, while accounting for the cost of computation, in addition to the number of multiplications and additions, the number of comparisons is also taken into consideration. By extending our previous notation, the computational complexity is expressed as αM +
βA + γC, implying α number of multiplications, β number of additions and γ number of comparisons. Computational costs for a multiplication and for a division are also considered as equivalent.
For n pixels, the grey-world (GRW) and the white-world (MXW) tech-niques, respectively, require 3M + 3(n − 1)A and 3(n − 1)C number of com-putations. For statistical techniques, it requires 9nM + 6nA operations for converting n pixels in the RGB color space to the YCbCr color space. These techniques also depend on the number of illuminants (say, nl) and the frac-tional coverage (say, f ) over the discretized chromaticity space. Let us consider the size of the discretized space as nc, which is mentioned as 32 × 32 in algo-rithmic descriptions (see Section 4.5.1). Hence, the number of computations for processing n pixels becomes f ncnlA + (nl− 1)C for both the techniques COR and GMAP, although the latter performs only integer additions. The NN technique computes the Mahalanobis distance with 8M + 4A operations for each illuminant, as it uses 2 × 1 chromatic mean and symmetric covariance matrices of dimension 2 × 2. Hence, the total number of computations for the NN becomes 8nlM + 4nlA for n pixels.
Every statistical technique needs to store the mean of the illuminant of dimension 3 × 1. Moreover, the NN technique needs to store three elements of the covariance matrix per illuminant. It makes its minimum storage re-quirement as 6nlnumber of elements. However, both the color-by-correlation (COR) and gamut-mapping (GMAP) techniques require more space for stor-ing the chromaticity map, which has ncnlnumber of elements. Additionally, they also need to store the mean illuminant (3 per illuminant).
In the block DCT space, color constancy techniques handle four 2×2 IDCT coefficients for each block. The computation of these values, as shown in Eq.
(4.14), requires 9 additions with only a marginal overhead in the computation.
Due to these operations, the input data size itself gets reduced by 14th of the size of the block, which is 16 times in the context of JPEG compression.
Hence, all the nonstatistical-based techniques run faster in the DCT domain.
However, the statistical techniques are relatively independent of the input data size. There are savings from the conversion of pixels in the RGB color space to the YCbCr color space. Moreover, in the compressed domain, the additional overhead of reverse and forward transforms to and from the spatial domain is also avoided in DCT-based approaches. A summary of computational and storage complexities of all the techniques is provided inTable 4.6.
4.5.2 Color Correction
If we get the estimates of the SPD of the illuminant, this becomes useful in transforming the pixel values to those under a target illuminant. This compu-tation is known as color correction and is performed following the Von Kries model [80]. The corresponding method is known as diagonal color correction, which is described in the following text. In the block DCT space, the operation is restricted to the DC coefficients only.
Table 4.6: Complexities of different algorithms given nl number of illumi-nants, nc as the size of the 2-D chromaticity space, and n number of image pixels
Algorithms Computational Storage
Complexity Complexity
GRW 3M + 3(n − 1)A –
GRW-DCT 3M + 3(4n+964 − 1)A –
MXW 3(n − 1)C –
MXW-DCT 649nM+24n+964 A+ 3(64n − 1)C –
MXW-DCT-Y (4n+964 − 1)C –
COR 9nM + (6n + f ncnl)A + (nl− 1)C ncnl+ 3nl
COR-DCT f ncnlA+ (nl− 1)C ncnl+ 3nl
GMAP 9nM + (6n + f ncnl)A + (nl− 1)C ncnl+ 3nl
GMAP-DCT f ncnlA+ (nl− 1)C ncnl+ 3nl
NN (9n + 8nl)M + (6n + 4nl)A 6nl
NN-DCT 8nlM+ 4nlA 6nl
Let the spectral components for the source illuminant be estimated as Rs, Gs, and Bs for red, green, and blue spectrum zones of the SPD, respectively.
Let the corresponding spectral components for the target illuminant be given by Rd, Gd, and Bd. From a source pixel in the RGB color space with R, G, and B as its corresponding color components, the transformed or color-corrected components (Ru, Gu, and Bu, respectively) are expressed as follows:
kr = RRd
s, kg = GGd
s, kb = BBd
s, f = krR+kR+G+BgG+k
bB,
Ru = f krR, Gu = f kgG, Bu = f kbB.
(4.16)
4.5.2.1 Color Correction in the YCbCr Color Space
The expressions for diagonal corrections (see Eq. (4.16)) in the RGB color space are translated in the YCbCr color space, as it deems to be appropri-ate for processing in the compressed domain. This is stappropri-ated in the following theorem.
Theorem 4.2 Let kr, kg, and kb be the parameters for diagonal correction as defined in Eq. (4.16). Given a pixel with color values in the YCbCr color space, the updated color values Yu, Cbu, and Cruare expressed by the following
equations:
Cb′= Cb− 128, Cr′ = Cr− 128,
f= 3.51Y +1.63Cb′+0.78C′r
1.17(kr+kg+kb)Y +(2.02kb−0.39kg)Cb′+(1.6kr−0.82kg)C′r,
Yu= f((0.58kg+ 0.12kb+ 0.30kr)Y + 0.2(kb− kg)Cb′+ 0.41(kr− kg)Cr′), Cbu= f((0.52kb− 0.34kg− 0.18kr)Y + (0.11kg+ 0.89kb)Cb′+
0.24(kg− kr)Cr′) + 128,
Cru= f((0.52kr− 0.43kg− 0.09kb)Y + 0.14(kg− kb)Cb′+ (0.3kg+ 0.7kr)Cr′) + 128.
(4.17)
However, there is no significant reduction in the number of multiplica-tions and addimultiplica-tions in Eq. (4.17) compared to the diagonal correction method applied in the RGB color space.
4.5.2.2 Color Correction by Chromatic Shift
In [39] a simple color correction method is proposed by translating the chro-matic components by the corresponding estimated components of the source illuminant. By adopting this simple strategy, it was demonstrated that good quality color rendition (or transfer) is possible. In this approach, if Yd, Cbd, and Crdare the color components of a target illuminant in the YCbCr space and the corresponding components in the source illuminant are Ys, Cbs, and Crs, color correction by chromatic shift is expressed by the following equations.
Yu = Y,
Cbu = Cb+ Cbd− Cbs, Cru = Cr+ Crd− Crs.
(4.18) Typical examples of color correction following the estimation of the illumi-nant using the color-by-correlation technique in the DCT domain (COR-DCT) are shown inFigure 4.8(see color insert).