Sintiendo indignación

models

methods sensitivity specificity precision accuracy f1 MCC

seed & mini & TCA-fast gap fill &

Bio-Growmatch

0.89 0.9 0.83 0.89 0.86 0.78

manual & mini 0.57 0.79 0.73 0.68 0.64 0.37

seed & full & TCA-fast gap fill &Bio-Growmatch

0.9 0.77 0.68 0.82 0.78 0.64

manual&self 0.14 0.96 0.79 0.56 0.24 0.18

For the automatically generated models the results are combined over 31 models. For the manual models the results are combined over 4 models.

Similarly, when the Medium biomass definition is used, the number of positive results is also zero for SEED generated models. However, models generated with PathoLogic show a positive, albeit small, number of positive predictions, which indicates a higher degree of model completion. Never- theless, even without gap-filling the automated methods are able to, at least partly, include catabolic pathways as shown by the relatively high accuracies obtained when using the Mini objective function. This is objective function is, as previously stated, an indicator of the catabolic capacities of the organisms detached from their anabolic capacities.

Default gap-filling methods included in the tested algorithms for auto- mated reconstruction result in an insubstantial improvement of the generated models, as shown in Table 10.2. This is most likely due to mismatches between the biomass definitions here presented and the approaches used by each of the algorithms.

Applying the anabolic fast gap-fill approach to the generated models leads to clear improvements in the predictive power of the models regarding the Medium and Full biomass objectives. MCC goes from zero up to ≈ 0.22 and ≈ 0.15 for models generated using SEED and PathoLogic respectively, regard- less of the selected biomass definitions (Table 10.2). These improvements are not surprising, given that this approach is specifically designed to improve the predictive power of the models regarding the anabolic potential of the species. These improvements are achieved at the cost of including a relatively large amount of reactions for which no genomic evidence is available (Table 10.1). SEED generated models seem to outperform the PathoLogic generated ones after anabolic fast gap-fill. Nevertheless, it should be noted that the num- ber of reactions that are added in the SEED generated models almost doubles the number of reactions the PathoLogic generated models require. The an- abolic fast gap-fill method was not intended to increase the predictive power of the model regarding catabolism, and, as expected, no modifications are in- troduced in the model (Table 10.1)

The gap-filling methods embedded in the reconstruction algorithms and the anabolic fast gap-filling methods result in addition of reactions to the

model, as these approaches work based on adding reactions that enable syn- thesis of biomass components. However, Bio-Growmatch aims at the maxi- mization of the agreement between model predictions and measured pheno- types. As a result, reactions are both added and removed from the model (Table 10.1). The more complex the biomass definition is, the more reactions are added and less reactions are removed, regardless of the originating model. For the Full biomass definition no reactions are removed. The resulting mod- els achieve MCC that are significantly better than the scores achieved with no gap-filling, the gap filling method included in Seed and PT, or the anabolic fast gap-fill method. In particular, Bio-Growmatch clearly improves the specificity of the models.

Finally, the best performing models arise as a result of the combination of Bio-Growmatch with the anabolic fast gap-filling method, which is not sur- prising given that this combination is the one that introduces the highest num- ber of modifications (added and removed reactions, as shown in Table 10.1). Bio-Growmatch is able to improve the models by reducing the number of in- correct positive predictions. However its applicability is limited when the starting model has such a high number of gaps that it is unable to simulate biomass synthesis and no positive predictions can be generated. Thus initial applications of the anabolic fast gap-filling approach improves the MCC and sensitivity scores, however it does so at the cost of specificity (Table 10.2).

Overall, Table 10.2 shows that, regardless of the starting model, applica- tion of Bio-Growmatch leads to models that better describe the metabolic po- tential of the studied organisms. Moreover, it also increases the predictive power of the model. This can be seen by comparing rows SEED/basic/ngf, SEED/basic/Bio-Growmatch and SEED/basic/AFGF&Bio-Growmatch in Ta- ble 10.2.

The comparison between manually curated models and those generated by Bio-Growmatch shows the surprising result of Bio-Growmatch outperform- ing the manually curated ones. However, caution should be exerted when in- terpreting this result. Bio-Growmatch has been designed to generate models able to account for metabolic phenotype data, even though in the carried on comparison the corresponding data was purposely left out of the process. On the other hand, published models have undergone a curation process that of- ten takes into account other data types, such as gene essentiality. Moreover, it should be noted that in the automatically generated models we have assumed the existence of transporters needed to uptake the corresponding compounds. Transport is often a major bottleneck in substrate utilization that in the gap- filling method has not been considered, whereas in manually curated models curation of transport mechanisms is often a major effort. Finally the score of the curated models can be hampered due to metabolite mapping issues. This is an intrinsic problem often found in the manual models, were unique and univoque metabolite identifiers for the metabolites, such as InChI identifiers [], are not included in the annotation.

10