Image restoration becomes image denoising when K becomes an identity matrix, then Equation 1.1 can be rewritten in its classical image denoising form, given an additive i.i.d. Gaussian noise model,
vec(y) =vec(x) +vec(n), (1.5) here, aim is to recover the clean image vec(x)∈ Rmfrom the noisy image vec(y)∈ Rm,
where vec(n) denotes the additive Gaussian noise with zero mean vector andσ2vari- ancei.e. vec(n)∼ N(0,σ2)∈ Rm. Figure 1.7 is an example for image denoising.
In Chapter 4 and Chapter 5, we address the image denoising using external im- ages. Similar to image deblurring, image denoising is also studied for decades; how- ever, it is still relevant as it is useful in many other many computer vision tasks such as object detection, classification, and tracking. Image denoising is also useful in many other image processing tasks, for example, image deblurring, image super- resolution, and image inpainting.
The introduction of noise in images may be due to many reasons, but two lead- ing causes stand out among all, which are electronic and shot noise. Electronic noise major causes are voltage instability, electronic components abrupt temperature
variations and analog to digital conversion. All these can be modeled as Gaussian distributed noise. Similarly, the random arrival of the photon on image sensor causes shot noise, and typically, it is modeled as a Poisson distribution. This noise is very challenging in low light condition.
Image denoising literature reveals that the noise is usually modeled as Additive White Gaussian Noise (AWGN) with zero mean due to two main reasons. Firstly, the Gaussian noise is practically applicable to other types of noises such as shot noise, as this can also be transformed into Gaussian noise using Anscombe root transformation [Anscombe, 1948]. Secondly, Gaussian distribution can facilitate the mathematical analysis as it is mathematically tractable.
With recent advancement in image denoising, researchers have started to investi- gate external priors [Zoran and Weiss, 2011; Yue et al., 2015] as opposed to internal priors [Dabov et al., 2007b; Buades et al., 2005]. The difference between external and internal stems from the fact that whether the reference patches are taken from the im- age itself or an external database. It is observed that internal priors are efficient and computationally less expensive whereas external priors achieve better performance. Our method discussed in Chapter 4 falls in external prior category.
Recently, image denoising also started taking advantage of CNN. The input to the network is a noisy observation while the target is the original clean image. Many works [Zhang et al., 2017a; Lefkimmiatis, 2016] are presented in this category and are growing to date. In Chapter 5 of this thesis, we discuss image denoising using CNN, and aim to take advantage of CNN and use external images to train our network and learn an external prior in a systematic way.
1.2.2.1 Limitation of Existing Denoising Algorithms
Here, we present common limitation of the existing state-of-the-art methods and then conclude image denoising in this chapter with our contributions to overcome the mentioned limitations.
• Many state-of-the-art algorithms [Dabov et al., 2007b; Buades et al., 2005] rely on internal self-similar patches to denoise the image. There are two main chal- lenges to image denoising: 1) internal image denoising methods are reaching its optimal performance [Levin and Nadler, 2011; Chatterjee and Milanfar, 2012], 2) the patches that rarely occur in the image, this “rare patch" effect causes the performance to decrease.
• Recently, CNN models are employed in image denoising. Undoubtedly, CNN based image denoising methods have proved to be superior regarding per- formance compared to the state-of-the-art classical methods. However, these
approaches still rely on the hyper-parameter settings, extensive fine-tuning, nonlocal self-similar patches, stage-wise training and learning noise pattern without exploiting the underlying structure. These elements impede the per- formance of CNN based image denoising.
1.2.2.2 Our Contributions to Denoising Task
To address and overcome the limitation of previous algorithms, we introduce new novel algorithms for image denoising. Our algorithms show superior performance compared to current competitive methods. In the following paragraphs, we list our contributions and state how we addressed the shortcomings of the competitive tech- niques.
• We present a novel category-specific image denoising algorithm that exploits patch similarity between the input image and an external dataset only. We rely on external images in the same category as the input, to denoise textured regions. The external denoising component estimates the latent patches using the statistics, i.e. means and covariance matrices, of external patches, subject to a low-rank constraint. We show that our algorithm let us handle of a large variety of categories.
• We propose to learn a fully-convolutional network model for image denois- ing. Our denoising model learns the noise with the underlying patch struc- ture. Also, we do not require stage-wise training and hyper-parameter setting. Our denoising network possesses distinctive features that are important for the noise removal task.
– Each residual unit employs identity mappings as the skip connections and receives pre-activated input to preserve the gradient magnitude propa- gated in both directions.
– Utilizing dilated kernels for the convolution layers in the residual branch, in other words within an identity mapping module, each neuron in the last convolution layer can observe the full receptive field of the first layer.
We have evaluated these novel algorithms on a number of datasets. We provide an extensive experimental evaluation at the end of each chapter which confirms that all our solutions surpass the performance of existing state-of-the-art methods quan- titatively and qualitatively.