Results and Discussion

Tuning parameter settings (Tables 5.2 and 5.3) were varied systematically for both algorithms. For RF, the size of the predictor subset (m, Table 5.2) was allowed to vary from 1 to the total number of available predictor variables. Simultaneously, the number of iterations varied from 500 to 8000 (T, Table 5.2). In the case of BRT, the learning (or shrinkage) rate (λ) took on a value of either 0.01 or 0.001; interac- tion depth (or the number of splits, K) was allowed to vary from 1 to the total num- ber of available predictor variables; and the number of iterations was varied from 500 to 8000 (T, Table 5.3).

Ensemble (ENS) predictions were generated, at each iteration of model construc- tion, by calculating a weighted average of the predictions from RF and BRT. The weightings were based on model performance using the cross-validation data, and were defined as the inverse of the MSE (MSE⁻¹).

Fig. 5.2 Comparative performance of random forests (RF), boosted regression tree (BRT), and ensemble (ENS) predictions, as measured using cross-validated mean square prediction error (MSE). Tree size was allowed to vary from 500 to 8000 trees (T = 8000 not shown), and tree depth/

size of predictor subset was set from 1 to 13. In the case of BRT, a shrinkage (λ) value of 0.01 was used

Fig. 5.3 Comparative performance of random forests (RF), boosted regression tree (BRT), and ensemble (ENS) predictions, as measured using cross-validated mean square prediction error (MSE). Tree size was allowed to vary from 500 to 8000 trees (T = 8000 not shown), and tree depth/

size of predictor subset was set from 1 to 13. In the case of BRT, a shrinkage (λ) value of 0.001 was used

All algorithms experienced highest cross-validated accuracies when lower numbers of covariates were specified - either in terms of the number of splits permissible to BRT trees (K) or the size of the predictor subset (m) for RF. Other authors have reported similar findings. For example, while Prasad et al. 2006 employed a MARS algorithm (Friedman 1991), they reported limitations in the portability of MARS predictions to future climate when more interaction terms were introduced in the model training stage, describing them as more “wild” (Prasad et al. 2006: 197).

The BRT and RF equivalent to increasing interaction terms is by increasing m (RF) or K (BRT). Elith et al. (2008, 807) suggest that larger data sets benefit more from this increased complexity, but we demonstrate that this can come at the price of increased prediction bias. As a final note, James et al. (2013, 320) argued that lower values of m are warranted when there is strong correlation amongst the covariates.

The effect of varying the numbers of constructed trees (T) differed depending upon the algorithm; in the case of BRT, cross-validated accuracy tended to be opti- mal for intermediate numbers of trees (i.e., T ≤ 3000) whereas it seemed less impor- tant for RF provided that m was set to one. By virtue of the additive nature in which BRT fits trees to increasingly smaller portions of residual variation (James et al.

2013), the judicious choice of parameter settings seems particularly important in order to prevent overfitting.

Fig. 5.4 Results for random forests out-of-bag (OOB) error assessment, as a function of the size of the predictor subset (m, or mtry in package randomForest of Liaw and Wiener 2002).

A commonly cited decision rule is based on the square-root of the number of predictor variables which, in this case, would be 3.6 (~4)

Our results demonstrate that for at least part of the “parameter space”, ensemble predictions yielded the lowest prediction error and, at worst, tended to track BRT performance with less variation in performance. In general, variation in the number of trees used to train RF models seemed unimportant, but for this particular dataset, cross-validated error was lowest when tree depth was set to low values. We propose that the strengths of the two be merged using an “ensemble of ensembles” such as we have done here. The impact of producing an “ensemble of ensembles” was three-fold: improved predictive accuracy, particularly when comparing ensemble predictions to those based on RF; somewhat of a dampening of variation in perfor- mance from iteration-to-iteration; and lower prediction bias under a range of condi- tions, as evidenced by the narrowing of the gap between optimistic and crossvalidated accuracy assessments.

A pertinent question to ask is why, despite a growing body of applications of ML methods in the ecohydrological (Peters et al. 2007), marine (Leathwick et al. 2006;

Pinkerton et al. 2010; Huettmann et al. 2011; Huettmann and Schmid 2015; Schmid et  al. 2016) and terrestrial (Cutler et  al. 2007; Jiao et  al. 2014; Mi et  al. 2014;

Baltensperger and Huettmann 2015) ecological literature, are ML techniques not more widely used by ecologists? It may be a result of less familiarity (Olden et al. 2008), or perhaps they are perceived as “black boxes” that are harder to interpret (Elith et al.

2008). We hope that these examples, along with our analysis, will continue to make a case for routine use of ML methods such as BRT and RF under the proviso that idiosyncrasies of particular data sets may render it difficult to determine, in advance, the optimal set of tuning parameters to guide the model building process.

Given the complexity of real-world ecological problems, and the difficulty in assessing, a priori, appropriate model structures to test or make predictions means that ML methods should be routinely used. They can be used to explore patterns, evaluate the impact of different predictor variables, and provide predictions in a standalone form or as part of an ensemble of other predictions. They are ideally suited for “mining” large, complex data sets, especially when little prior knowledge about the system exists (Hochachka et al. 2007). Our results demonstrate that the combination of predictions from multiple algorithms, to form ensemble or consen- sus predictions, is straightforward to implement and can result in higher predictive accuracy than the results from single algorithms alone. Ensemble predictions have the added benefit of reducing the reliance on single techniques and allowing a wider range of potentially useful algorithms to be employed.

References

Araújo MB, New M (2007) Ensemble forecasting of species distributions. Trends Ecol Evol 22:42–47

Baltensperger A, Huettmann F (2015) Predictive spatial niche and biodiversity hotspot models for small mammal communities in Alaska: applying machine-learning to conservation planning.

Landsc Ecol 30:681–697

Brawn C (2016) Marine traffic risk density in Atlantic Canada. Unpubl. Report

Breiman L (2001) Random forests. Mach Learn 45:5–32

Breiman L (2002) Manual on setting up, using, and understanding random forests v3.1. https://

www.stat.berkeley.edu/~breiman/Using_random_forests_V3.1.pdf. Accessed 31 July 2015 Caruana R, Niculescu-Mizil A (2006) Proceedings of the 23rd international conference on machine

learning, Pittsburgh, PA

Clemen RT (1989) Combining forecasts: a review and annotated bibliography. Int J  Forecast 5:559–583

Cutler DR, Edwards TC Jr, Beard JH, Cutler A, Hess KT, Gibson J, Lawler JL (2007) Random forests for classification in ecology. Ecology 88:2783–2792

Das SK, Chen S, Deasy JO, Zhou S, Yin F-F, Marks LB (2008) Combining multiple models to generate consensus: application to radition-induced pneumonitis prediction. Med Phys 35:5098–5109

De’ath G, Fabricius KE (2000) Classification and regression trees: a powerful yet simple technique for ecological data analysis. Ecology 81:3178–3192

Domingos P (2012) A few useful things to know about machine learning. Commun ACM 59:78–87 Elith J, Burgman M (2002) Predictions and their validation: rare plants in the central highlands,

Victoria, Australia. In: Scott JM, Heglund PJ, Morrison ML, Raphael MG, Wall WA, Samson FB (eds) Predicting species occurrences: issues of accuracy and scale. Island Press, Covelo, pp 303–314

Elith J, Graham CH (2009) Do they? How do they? WHY do they differ? On finding reasons for differing performances of species distribution models. Ecography 32:66–77

Elith J, Graham CH, Anderson RP, Dudik M, Ferrier S, Guisan A, Hijmans RJ, Huettmann F, Leathwick JR, Lehmann A, Li J, Lohmann LG, Loiselle RA, Manion G, Moritz C, Nakamura M, Nakazawa Y, Overton JM, Peterson AT, Phillips SJ, Richardson K, Scachetti-Pereira R, Schapiro RE, Soberón J, Williams S, Wisz MS, Zimmermann NE (2006) Novel methods improve prediction of species’ distributions from occurrence data. Ecography 29:129–151 Elith J, Leathwick JR, Hastie T (2008) A working guide to boosted regression trees. J Anim Ecol

77:802–813

Fernández-Delgado M, Cernadas E, Barro S, Amorim D (2014) Do we need hundreds of classifiers to solve real-world classification problems? J Mach Learn Res 15:3133–3181

Franklin J (1995) Predictive vegetation mapping: geographic modelling of biospatial patterns in relation to environmental gradients. Prog Phys Geogr 19:474–499

Friedman JH (1991) Multivariate adaptive regression splines. Ann Stat 19:1–67

Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29:1189–1232

Harrell FE (2001) Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. Springer, New York

Hastie T, Tibshirani R, Friedman J (2001) The elements of statistical learning: data mining, infer- ence, and prediction. Springer, New York

Hegel TM, Cushman SA, Evans J, Huettmann F (2010) Current state of the art for statistical modelling of species distributions. In: Cushman SA, Huettmann F (eds) Spatial complexity, informatics, and wildlife conservation. Springer, New York, pp 273–312

Heikkinen RK, Luoto M, Araújo MB, Virkkala R, Thuiller W, Sykes MT (2006) Methods and uncertainties in bioclimatic envelope modelling under climate change. Prog Phys Geogr 30:751–777

Hochachka WM, Caruana R, Fink D, Munson A, Riedewald M, Sorokina D, Kelling S (2007) Data- mining discovery of pattern and process in ecological systems. J Wildl Manag 71:2427–2437 Huettmann F, Artukhin Y, Gilg O, Humphries G (2011) Predictions of 27 Arctic pelagic seabird

distributions using public environmental variables, assessed with colony data: a first digital IPY and GBIF open access synthesis platform. Mar Biodivers 41:141–179

Huettmann F, Schmid M (2015) Climate change predictions of pelagic biodiversity components.

In: De Broyer C, Koubbi P, Griffiths HJ, Raymond B, Udekem d’Acoz C, Van de Putte AP, Danis B, David B, Grant S, Gutt J, Held C, Hosie G, Huettmann F, Post A, Ropert-Coudert Y

(eds) Biogeographic Atlas of the Southern Ocean. Scientific Committee on Antarctic Research, Cambridge, pp 390–396

James G, Witten D, Hastie T, Tibshirani R (2013) An introduction to statistical learning: with applications in R. Springer, New York

Jiao S, Guo Y, Huettmann F, Lei G (2014) Nest-site selection analysis of Hooded Crane (Grus monacha) in northeastern China based on a multivariate ensemble model. Zool Sci 31:430–437

Leathwick JR, Elith J, Francis MP, Hastie T, Taylor P (2006) Variation in demersal fish species richness in the oceans surrounding New Zealand: an analysis using boosted regression trees.

Mar Ecol Prog Ser 321:267–281

Liaw A, Wiener M (2002) Classification and regression by Random forest. R News 2(3):18–22 Lieske DJ, Fifield DA, Gjerdrum C (2014) Maps, models, and marine vulnerability: assessing the

community distribution of seabirds at-sea. Biol Conserv 172:15–28

Lieske DJ, Schmid M, Mahoney M (2018) Data and analysis script. https://doi.org/10.5281/

zenodo.1318352

Marmion M, Luoto M, Heikkinen RK, Thuiller W (2009a) The performance of state-of-the-art mod- elling techniques depends on geographical distribution of species. Ecol Model 220:3512–3520 Marmion M, Parviainen M, Luoto M, Heikkinen RK, Thuiller W (2009b) Evaluation of concensus

methods in predictive species distribution modelling. Divers Distrib 15:59–69

Mi C, Huettmann F, Guo Y (2014) Obtaining the best possible predictions of habitat selection for wintering Great Bustards in Cangzhou, Hebei Province with rapid machine learning analysis.

Chin Sci Bull. Published online: https://doi.org/10.1007/s11434-014-0445-9

Moisen GG, Freeman EA, Blackard JA, Frescino TS, Zimmermann NE, Edwards TC (2006) Predicting tree species presence and basal area in Utah: a comparison of stochastic gradient boosting, generalized additive models, and tree-based methods. Ecol Model 199:176–187 Olden JD, Lawler JJ, Poff NL (2008) Machine learning methods without tears: a primer for ecolo-

gists. Q Rev Biol 83:171–193

Oppel S, Meirinho A, Ramírez I, Gardner B, O'Connell AF, Miller PI, Louzao M (2012) Comparison of five modelling techniques to predict the spatial distribution and abundance of seabirds. Biol Conserv 156:94–104

Peters J, De Baets B, Verhoest MEC, Samson R, Degroeve S, De Becker P, Huybrechts W (2007) Random forests as a tool for ecohydrological distribution modelling. Ecol Model 207:304–318 Pinkerton MH, Smith ANH, Raymond B, Hosie GW, Sharp B, Leathwick JR, Bradford-Grieve JM (2010) Spatial and seasonal distribution of adult Oithona similis in the southern ocean: predic- tions using boosted regression trees. Deep-Sea Res I 57:469–485

Prasad AM, Iverson LR, Liaw A (2006) Newer classification and regression tree techniques:

bagging and random forests for ecological prediction. Ecosystems 9:181–199

Renner M, Parrish JK, Piatt JF, Kuletz KJ, Edwards AE, Hunt GL Jr (2013) Modeled distribution and abundance of a pelagic seabird reveal trends in relation to fisheries. Mar Ecol Prog Ser 484:259–277

Ridgeway G (2012) Generalized boosted models: a guide to the gbm package. URL: http://gra- dientboostedmodels.googlecode.com/git/gbm/inst/doc/gbm.pdf (downloaded January 8, 2016) Salford Systems, Inc. (2016) https://www.salford-systems.com/

Schmid MS, Aubry C, Grigor J, Fortier L (2016) The LOKI underwater imaging system and an automatic identification model for the detection of zooplankton taxa in the Arctic Ocean.

Methods Oceanogr In pres

123

© Springer Nature Switzerland AG 2018

G. R. W. Humphries et al. (eds.), Machine Learning for Ecology and Sustainable Natural Resource Management, https://doi.org/10.1007/978-3-319-96978-7_6