I further trained the image classification with the region labeling hints by jointly optimizing the loss for the labeling hints Clabelingand for the meta-task Cclassification together as a joint loss:
minimize
θ whClabeling+ (1−wh)Cclassification (5.1)
whereθrepresents the set of all parameters andwh ∈[0,1]is the guided ratio, which is set to 1 at the
beginning of the training and linearly decreased to 0.1 in 10,000 updates. Ifwh = 0, the objective is
equivalent toCclassification, and equals toClabelingaswh = 1. I found that jointly trained on the hint
model and the classification model can improve the ultimate classification results significantly. The optimization process concentrates on different objectives according to the weight wh.
Early in the process it focuses on getting the labeling model correct, and as the process goes on, it emphasizes the accuracy of the classification model. I gathered two datasets for both models and let two models learn jointly, and the results can attain higher than 85% accuracy as shown in Table 5.3(d).
Figure 5.12 presents some samples from the generated Brodatz dataset and their corresponding false-color results indicating the label with the highest probability at each pixel. Figure 5.12(a) are the examples with the same foreground and background textures. From the observation of the false-color images, although the model has been misled by some distinct features represented in the same image which results in identifying two different labels, the model is able to recognize different textures for most samples. On the other hand, Figure 5.12(b) shows the examples with different textures for the foreground and background. The labeling model generally can identify different textures and the borders between the textures in some examples are clearly discovered.
(a)
(b)
Figure 5.12: (a) Samples from generated dataset with the same foreground and background textures and their false-color images indicating the label with the highest probability at each pixel. (b) Samples and their false-color images with different foreground and background textures.
As a result, the experiments presented in this section suggest that a hard to learn problem can be made simple if we can guide the model to learn in the right direction through curriculum learning. It also shows that a labeling model can be generalized to different dataset.
5.4 Conclusion
In this chapter, I applied my proposed labeling model to two different datasets, a classic Pentomino dataset and a new Brodatz texture dataset, to demonstrate the generalization of the labeling model I have developed. In the Pentomino toy problem, the experiments show that my labeling model can provide meaningful hints to different subsequent tasks, P1NN and P2NN, and improve the performance significantly. Although the pixel labeling hints are not as good as SMLP-Hints in 40k dataset, they still proved to be useful and a more practical classifier in the real world. Also, the sensitivity experiments prove that the pixel labeling classifier is robust and not susceptible to the noise.
In addition, I presented a new toy problem based on Brodatz texture Album, which, I believe, is more similar to the histology image scenario. The experiments imply that the labeling model can provide valuable hints for the ultimate classification goal. Even if the model is not directly learned with hints but initialized with the weights learned from the supervised labeling model, the accuracy can still be increased by 30%. Furthermore, optimizing the model by jointly learning the labeling hints and the meta-task improves the performance even more significantly.
In conclusion, I evaluated the labeling approach on two toy problems, the experiments suggest that the advantages of a pixel labeling classifier are threefold. First, it generalizes to other related vision tasks. The experiments in two toy problems show that the classification performance is improved with the labeling hints. Second, it is robust and not susceptible to pixel label noise. The labeling approach serves as an intermediate hint for the meta task. Image classification does not require extremely accurate pixel labeling. Third, it is beneficial for classifying a texture-like dataset. The labeling model uses a small aperture window to analyze the elements composed in an object.
The low-level texture information might be lost while training goes deeper in the hierarchy, but the labeling model captures them and converts them to valuable hints.
CHAPTER 6: UNSUPERVISED PIXEL LABELING LAYER
6.1 Introduction
In Chapter 4, pixel/region labeling proved to be useful and practical as an intermediate hint, and I also showed that it generalizes to different problems in Chapter 5. Still, one might argue that the need for an extra training dataset for the pixel labeling model makes the hint augmented training less appealing. This observation motivates the question of whether is there a way to construct a labeling model with the benefits of the hint layer, but without explicitly providing hints.
An unsupervised labeling model might make this additional training process unnecessary. First, because no ground-truth labels are required during training process, no extensive prior knowledge is required. Second, the hint labels are generated purely based on the data, and therefore they are not subjective to visual interpretation (Bengio, 2012; Berg-Kirkpatrick et al., 2010; Xie et al., 2016; Dizaji et al., 2017).
However, optimizing an unsupervised deep learning model is challenging. The model has to learn underlying feature representations and assign clusters simultaneously. Without additional labeled training data, the model has to iteratively refine the label assignments with an auxiliary learning objectives for shaping the output distribution to gradually improve the labeling results as well as refine the feature representations (Bengio, 2012; Xie et al., 2016).
In this chapter, I propose an approach which extends the architecture introduced in Chapter 4. The classification task is divided into two subtasks, a low-level labeling and an image-level classification. The labeling model is trained by a CNN, whose output is a probability of labels. The output of the labeling model serves as an intermediate hint and is used as the input of image classifier for further training.
I aim to investigate the potential for an unsupervised learning objective which includes three regularization terms. The terms are inspired by the observations of the supervised labeling results, and are designed to constrain the shape of the output distribution. They implicitly aid the unsu- pervised model in learning feature representations and ultimately output cluster-like labels for the downstream learning task.
6.2 Approach
In this section, I introduce three penalization terms to the output of the labeling model based on the desired property of its probability distribution. The approach is studied and evaluated on Brodatz texture dataset introduced in Section 5.3. Section 6.2.1 examines the importance of the probability distribution of the pixel labeling output to the unsupervised loss function. In addition, inspired by an approach proposed by Kilinc and Uysal (2017), I present three regularization terms, uncertainty, sparsity, and distinction, to form the learning objective for the unsupervised labeling model. These terms enforce or penalize correlation relationships between label distributions throughout the unsupervised learning, which is described in Section 6.2.2. Finally, the details of three regularization terms along with the relationships to other measures are detailed in Section 6.2.3.