Impacto de las redes sociales

In this subsection, we provide a quantitative evaluation of the quality of the artificial image generation process. A qualitative analysis of the artificial patches with regard to their usability for the generation of accurate and reliable tie points through traditional matching approaches is provided in Subsection5.2.3. All artificially generated images patches shown in this and the following subsections are obtained from the set of test image patches, and hence have never been shown to the different generator networks during the training process.

SAR image generation: We first investigate the generation of artificial SAR and despeckled SAR image patches from optical images. Figure5.4 shows examples of (despeckled) SAR patches with a pixel spacing of 2.5 m generated by two different generators. The first generator, utilized for the SAR image generation, was trained with the cWGAN loss, a batch size of one and on the smaller training dataset. The second generator, utilized for the despeckled SAR image generation, was trained with the cGAN loss, a batch size of 40 and on the smaller dataset, where the filtered SAR images were used as reference. These two configurations led, from a visual point of view, to the most realistic-looking SAR and despeckled SAR image patches. The illustrated examples show that the geometric structures of streets extracted from optical images are preserved in the generated patches, while the radiometric properties are adapted to SAR or despeckled SAR images. Through the training process the generators learned that in contrast to optical images, streets normally appear with a lower intensity in SAR images. Furthermore, the generators try to represent the characteristics of speckle or the resulting pattern from the speckle filter. The development of the learning process of both generators trained with the two different configurations over the training time are exemplified in Figure 5.5. The longer we trained the networks, the better become the generators in generating realistic looking (despeckled) SAR images from optical image patches.

Despite the good visual appearance of the imitated texture of the speckle and the speckle filter, it is important to note that it is randomly generated and independent from the real image objects or their properties. Furthermore, the (despeckled) SAR image generation is not free of errors and in some situations the generators produces unsatisfying results (see Figure5.6). A possible reason for the difficulties of the networks in generation image patches

102 5. Results and Discussion

optical image artificial image SAR image optical image artificial image SAR image

SAR generation desp ec kled SAR generation

Figure 5.4:Side by side comparison between optical, artificial (despeckled) SAR and real (despeckled) SAR image patches with a pixel spacing of 2.5 m in two columns. SAR generation: The generator used to generate the artificial SAR images was trained with the cWGAN loss, a batch size of one and on the smaller training dataset. Despeckled SAR generation: The generator used to generate the artificial despeckled SAR images was trained with the cGAN loss, a batch size of 40 and on the smaller dataset with filtered SAR images as reference.

for the runway example (first row and column) is the small amount of runway patches in the training dataset. This problem could be solved with a larger runway training dataset or a separated training of street and runway patches. In the other three cases it can be seen that some features are present in the optical images but are missing in the generated images. Since optical images exhibit a higher level of detail than the SAR images the network learns during the training to ignore some of the features/objects for the generation of (despeckled) SAR images. However, for our later application it is essential that features that are valuable

5. Results and Discussion 103

optical image epoch 1 epoch 10 epoch 50 epoch 200 SAR image

SAR generation desp ec kled SAR generation

Figure 5.5: Development of the generator over training. From left to right: optical input patches, the artificially generated patches at epoch 1, 10, 50, 200 and the (despeckled) SAR target patches. The first two rows show the development of a generator trained for the generation of SAR patches by using the cWGAN loss, a batch size of 1 and the smaller training dataset. The third and fourth rows show the development of a generator trained for the generation of despeckled SAR patches by using the cLSGAN loss, a batch size of 4 and the larger training dataset with the filtered SAR images.

for the image matching, e.g. street and street crossings, are still present in the generated images. A possible solution for this problem could be to adjust the training procedure by adding the actual problem, the matching between the generated images and the reference image, into to training objective. Thereby the generator would learn, which features are crucial for the matching process, and hence which features should be kept in the artificial image generation process.

Optical image generation: We further investigated the reversed process and therefore trained networks in order to generate artificial optical images out of SAR images. Examples of such artificially produced optical images are shown in Figure 5.7. The corresponding generator was trained using the cGAN loss, a patch size of 4 and over the large training dataset. This configuration led (from a visual point of view) to the best and most realistic looking artificial optical images. Similar to the (despeckled) SAR image generation, the generator learned to keep the geometric structures of objects such as streets from the SAR images, while adapting the radiometric properties to optical images. Figure5.8shows two samples that illustrate the learning process of the generator over the training time.

104 5. Results and Discussion

optical image artificial image SAR image optical image artificial image SAR image

Figure 5.6: Comparison of failure cases of artificially generated SAR images with optical and real (despeckled) SAR image patches. The first row shows low quality artificial SAR images, and the second row low quality artificial despeckled SAR images.

Like for the SAR image generation, the learned generator model is not perfect and provides for some input images optical images of low quality (see Figure 5.9). Due to the lower level of detail in SAR images and the speckle it is more difficult to extract and recreate features from SAR than from optical images. Most of the details are missing in the SAR images and a realistic recreation is therefore almost impossible for the networks. As a consequence, the network tries to come as close as possible to real optical images by adding additional structure to the artificially generated images. These created structures might look realistic but is not derived from the input images.

SAR image artificial image optical image SAR image artificial image optical image

Figure 5.7:Comparison between SAR, artificial optical and real optical image patches. The generator used to generate the artificial optical images was trained with the cGAN loss, a batch size of 4 and on the larger training dataset.

5. Results and Discussion 105

SAR image epoch 1 epoch 10 epoch50 epoch 200 optical image

Figure 5.8: Development of the generator over training. From left to right: the SAR input patches, the artificially generated patches at epoch 1, 10, 50, 200 and the optical target patch. The generator used to generate the artificial optical images was trained with the cGAN loss, a batch size of 4 and on the larger training dataset.

SAR image artificial image optical image SAR image artificial image optical image

Figure 5.9: Comparison of failure cases of artificially generated optical images with real optical and SAR image patches.

In general, the concept of cGANs (introduced in Subsection4.2) enables the translation from SAR to optical images and vice versa. In both directions, realistic looking optical and SAR images respectively can be generated. In practice, the cGAN and cWGAN loss led to more realistic looking images compared to the cLSGAN loss (see Figure5.10for a comparison). However, for our pursued application it is not important that the obtained images look real. The important aspect is that the artificially generated patches improve the quality of the matching between optical and SAR images. Therefore, a detailed investigation and discussion about the effects of the different configuration, e.g. the three losses and the kind of input and reference data (SAR, despeckled or optical), on the quality of the generated tie points follows in the next subsection.

106 5. Results and Discussion