3.2 Methods based on Deep Learning
3.2.2 Unsupervised Learning Methods
CHAPTER 3. Related Work
3.2. Methods based on Deep Learning intensities.
Uzunova et al.(2019) have the assumption that blurred reconstructed images are due to the training stage just including MSEas loss function. Under this assumption, conditional VAE is introduced to detect pathologies in 2D and 3D CT images, which also includes theKL-divergence loss, andL1 loss for the training stage. In the inference stage, an anomaly score based on MSE is used as in the previous work. This change in the training stage achieves less smooth reconstructed images. In the same way, a VAE is proposed to improve the anomaly localization (Zimmerer et al., 2019). To this end, the KL-divergence is integrated into a pixel-wise anomaly score. However, this approach demonstrates that VAE is prone to blurred reconstructed images at a higher level than conditional VAE.
CHAPTER 3. Related Work
Table3.3:Publicationsbasedonunsupervisedlearning:AutoencodersandVariationalAutoencoders. PublicationsMethodDatasetQuantitativeMeasure Bermudezetal.(2018)Skip-connected autoencoders (Denoising Autoencoder).
T1-weightedbrainMRI. PeakSignal-to-Noise Ratio(PSNR). Pawlowskietal. (2018)Bayesian autoencoders.BrainCTmid-axialslices.AUC. Satoetal.(2018)3D convolutional autoencoder.
BrainMRI.Sensitivity,Specificity, AUC. Uzunovaetal.(2019)Conditional VAE.BrainMRIfromthe MultimodalBrainTumor ImageSegmentationBenchmark (BRATS)2015challenge.
Sensitivity,Specificity, DICE,AUC. Zimmereretal.(2019)VAE.T1andT2weightedMRI,the BRATS2017challenge.Sensitivity,Specificity, DICE,AUC.
Master Program in Computer Science - UCSP 45
3.2. Methods based on Deep Learning 3.2.2.2 Generative Adversarial Networks
The generative adversarial network introduced byGoodfellow et al.(2014) is a relatively new approach that belongs to the generative model family with significant progress. In spite of its potential performance in unsupervised tasks, GAN suffers from two main limitations. First, there is no guarantee of balance between the training of G and D.
As a consequence, one network may inevitably be more powerful than the other, which in most cases is D (Yi et al.,2019). Second, GANis prone to mode collapse problems, which is the case when the distribution learned by G focuses on a few limited modes of the data distribution. Hence, instead of producing diverse images, it generates a limited set of samples (Lin et al.,2018; Srivastava et al.,2017).
In order to stabilize training and also to avoid mode collapse several variants were proposed based on varying the objective of D, varying the objective of G and varying the architecture of the originalGAN(Yi et al.,2019). In this regard, there are generally two ways GANs are used in medical imaging. The first is focused on the generative aspect, which can help in exploring and discovering the underlying structure of training data and learning to generate new images. The second focuses on the discriminative aspect, where discriminator D can be regarded as a learned prior distribution for normal images so that it can be used as regularizer or detector when presented with abnormal images (Yi et al.,2019; Litjens et al., 2017; Schlegl et al.,2017).
Based on the generative aspect,Yi and Babyn(2018) introduced Sharpness-aware generative adversarial network (SAGAN), a pretrained sharpness detection network to explicitly constrain the sharpness of the denoised CT especially for low contrast regions. Wolterink et al. (2017) propose to train a GAN with unpaired MRI and CT images. This proposedGAN consists of twoCNNs for synthesis and twoCNNs for the discriminator, which were trained with cycle consistency to transform 2D brain MRI slices into 2D brain CT image slices and vice versa.
On the other hand, based on the discriminative aspect, we can find abnormality detection applications in the majority of cases. Wei et al. (2018) propose an autoencoder based on deep convolutional neural networks to learn to differ the manifold from the normal patches as in (Schlegl et al., 2017). Schlegl et al. (2017) proposed AnoGAN to identify abnormal images and to segment abnormal regions into Optical Coherence Tomography (OCT). The basic idea is to learn to differ the manifold from the normal anatomical data variability using Deep Convolutional Generative Adversarial Network (DCGAN) (Radford et al., 2015) and apply a novel anomaly scoring based on the fitness of the latent space of a test image to the learned manifold by D.
In order to improve the AnoGAN results, Schlegl et al. (2019) proposes Fast AnoGAN (f-AnoGAN), which uses a Wasserstein GAN (WGAN) architecture instead of DCGAN architecture (see Figure 3.1). It allows to speed up the mapping of input images to the latent space by moving from an iterative gradient descent approach to a learned mapping. On the other hand, Alex et al. (2017) propose the use of a GAN approach for brain lesion detection on MRI. In this regard, the generator was used
CHAPTER 3. Related Work
to model the distribution of normal patches and the trained discriminator was used to compute a posterior probability of patches centered on every pixel in the test image.
In spite of the success of these approaches, they work with patches; this involves a lot of redundancy due to their not controlling the overlapping of patches.
Figure 3.1: Anomaly detection framework. Both steps of model training, generative adversarial training (yields a trained generator and discriminator) and encoder training (yields a trained encoder), are performed on normal (healthy) data and anomaly detection is performed on both unseen healthy cases and anomalous data (Schlegl et al.,2019).
Baur et al. (2018) propose the AnoVAEGAN method in order to segment the abnormal region in brain MRI. The model is able to learn the shape and texture information from normal images. Hence, when an abnormal image is tested, the model is able to reconstruct it in a healthy version, then applies a simple subtraction between the original image and the reconstructed image, to segment the abnormal region. Although this method demonstrates that AnoVAEGAN is able to reconstruct realistic brain MRI, the DICE score is not good enough compared with other works.
We believe that it is due to the reconstructed images not preserving all the features from the input images; in other words, the reconstructed images are not the same as the input images due to a lack of consistency in the latent space representation.
Only a few works in the literature demonstrate that GANs are able to detect anomalies in X-ray imaging. Tang et al. (2019) propose the use of GANs to detect abnormal chest X-rays considering the distribution of healthy images. In this regard, normal images will have a good reconstruction, and abnormal images will have a poor reconstruction. Another remarkable work to detect anomalies in X-ray imaging is proposed byAkcay et al.(2018). It develops a novel method denominated GANomaly to detect anomalies in baggage X-ray imaging. This approach finds that L1 loss can help G to learn contextual information about the input data. Moreover, Isola et al.
(2017), Xue et al.(2018), Ak¸cay et al.(2019), and Akcay et al. (2018) determine that L1 loss in the discriminator avoids blurry results. Even though it is a semi-supervised approach, it has the potential to be applied in an unsupervised manner with some changes in its architecture.
Considering the advantages and disadvantages of the related work, we propose an unsupervised GAN-based approach. In contrast with the close related methods such Master Program in Computer Science - UCSP 47
3.2. Methods based on Deep Learning as GANomaly, which work on X-ray images, our approach does not require any kind of label. On the other hand, our approach is not limited to circle-like elements since it tries to take advantage of the discriminative aspect from GANto detect the anomalies, and from the generative aspect to localize the anomalies applying a simple subtraction as the AnoVAEGAN approach.
CHAPTER 3. Related Work
Table3.4:Publicationsbasedonunsupervisedlearning:GenerativeAdversarialNetworks. PublicationsMethodDatasetQuantitativeMeasure YiandBabyn(2018)SAGAN.LungCTsfromtheNational CancerImagingArchive, LungLowDoseComputed Tomography(LDCT)sfromthe DataScienceBowl2017.
PSNR,Structural SimilarityIndex (SSIM). Wolterinketal.(2017)CycleGAN.BrainMRIandCTimages,The Netherlands.MeanAbsoluteError (MAE),PSNR. Weietal.(2018)AEbasedon adeepneural network.
InBreastdataset,Porto.AUC. Schlegletal.(2017)AnoGAN.Spectral-domainOCTscans.Precision,Recall, Sensitivity,Specificity, AUC. Continuedonnextpage
Master Program in Computer Science - UCSP 49
3.2. Methods based on Deep Learning
Table3.4Publicationsbasedonunsupervisedlearning:GenerativeAdversarialNetworks. PublicationsMethodDatasetQuantitativeMeasure Schlegletal.(2019)f-AnoGAN.Spectral-domainOCTscans.Precision,Sensitivity, Specificity,f-score, AUC. Alexetal.(2017)GAN.BRATS,andIschemicLesion (ISLES)dataset.Accuracy. Bauretal.(2018)AnoVAEGAN.BrainMRI(inhousedataset).DICE. Tangetal.(2019)Generative adversarial one-class learning.
NIHClinicalCenterChestX-ray dataset.AUC. Akcayetal.(2018)GANomalyMNIST,CIFAR10,University BaggageAnomalyDataset.AUC.
CHAPTER 3. Related Work