3. HIPÓTESIS
3.3.3.1. Técnicas de recolección de datos
3.3.3.1.1. Observación Directa
Main Effects
Box plots showing the results from the 20 split permutations for the main effects can be seen in figures 4.1 and 4.2. There are many findings presented here, which will be discussed in turn. The first is that the performance for the even degrees of the polynomial kernels display the recognised drop in performance. The second main finding is that the performance of the polygenic score suffers when only subsets of the features are used to determine the phenotypes. This is perhaps not surprising as this score is meant to represent the combined contributions from each feature, as determined by large scale GWAS, so it is unrealistic to assume that large sets of the inputs would not make any contribution. Also, this polygenic score was made from all of the features, and then fixed, so was not flexible to adapt to the different inputs. But from a machine learning perspective, this really highlights the strength of using these models in that they are capable of finding the relevant features to make their classifications.
Probability 0.8 Probability 0.9 Probability 1 0.4 0.6 0.8 1.0 R OC Score
Algorithm
Polygenic Score Linear RBF Polynomial 2 Polynomial 3 Polynomial 4Figure 4.1.: Box plot showing the performance of the algorithms for phenotypes simulated from different levels of penetrance of the main effects. As the tight performance makes the colours difficult to see, the algorithms listed top to bottom in the legend go from left to right on the plots.
All Features Half of Features Third of Features
0.4 0.6 0.8 1.0 R OC Score
Algorithm
Polygenic Score Linear RBF Polynomial 2 Polynomial 3 Polynomial 4Figure 4.2.: The performances of the algorithms for simulated phenotypes made with different subsets of the main effects. As the tight performance makes the colours difficult to see, the algorithms listed top to bottom in the legend go from left to right on the plots.
Interactions
Similar plots to the main effects, showing the different performance of the algorithms on the prediction of the simulated phenotypes made from the pairwise interactions are shown in figures 4.3 and 4.4. These data reveal a very different pattern of behaviours, not seen in the main effect simulations or the results in chapter 3. As might be expected, the performance of the polygenic score and the linear kernel is very poor, at the chance level of 0.5. There is a clear improvement seen in non-linear kernels, but the RBF, and the polynomial-2 especially, are performing better. It makes logical sense that the polynomial-2 should be performing particularly well as it is based around looking for pairwise multiplicative interactions between the features - exactly the method used in these simulations. However, the performance is far from perfect, as even at the maximum probability level, it does not get better than 0.8 Receiver Operating Characteristic (ROC). As seen in the main effects examples, varying the probability levels results in the largest changes seen in performance. The models are once again quite resilient to there only be- ing subsets of the features used to make the interactions. This suggests that the Support Vector Machine algorithm is particularly good at identifying contributing features from those that are not providing any information and are therefore just nuisance variables in the models.
Probability 0.8 Probability 0.9 Probability 1
0.4 0.6 0.8 1.0 R OC Score
Algorithm
Polygenic Score Linear RBF Polynomial 2 Polynomial 3 Polynomial 4Figure 4.3.: Boxplot showing the performance of the algorithms for phenotypes simulated from different levels of probability of the interactions.
All Features Half of Features Third of Features 0.4 0.6 0.8 1.0 R OC Score
Algorithm
Polygenic Score Linear RBF Polynomial 2 Polynomial 3 Polynomial 4Figure 4.4.: The performances of the algorithms for simulated phenotypes made with different subsets of the input features. As the tight performance makes the colours difficult to see, the algorithms listed in the legend go from left to right on the plots.
Based on the findings of these simulations, it looks as though the pattern of results that most resembles that seen in the real data in chapter 3 is the performance seen on the phenotypes simulated from the main effects, using all of the input features. While the performance seen in the models does not exactly match that seen in the real data (as in the real data, it is the polygenic score which performs best and not the linear kernel), it is showing the very characteristic drops in performances for the even-valued polynomial kernels. Also, it is assumed from these results that all of the main effects should be considered when building the simulations. This is because, in the real dataset in the previous chapter, the polygenic score was always performing better than the SVMs. In these simulations, it can be seen that only providing a subset of the main effects has a detrimental outcome on the performance of the polygenic score, a pattern of behaviour not previously seen. In addition, it makes logical sense that all of the main effects should play a role, as they were selected due to their high association with schizophrenia from the GWAS results (Ripke et al, 2014).
Another interesting finding was that the RBF kernel seemed to show quite robust per- formance levels for both the main effects and the interactions. While the linear kernel was best for the main effects, and the polynomial-2 kernel was best for the pairwise in- teractions, both of these performed extremely badly when the conditions were swapped. The conclusion from this is that, if the desired outcome is to gain predictive performance without the need for interpretability of the model, the RBF kernel is a good first step, based on the versatile levels of performance.