I'm using biomod2 in R and my ensemble model performs worse on the evaluation data (drastically lower ROC, 0.835) than any of the individual models (ROC ranges 0.89-0.97). What could be causing this? Code example below. I'm using a 70/30 training/evaluation split as well. Note: ROC performance for training data is equivalent between individual models vs ensemble. Thank you!

myBiomodModelOutHyacinthBig <- BIOMOD_Modeling( chuck.sp.bm, 
                                 models = c('GLM', 'GAM', 'RF', 'GBM'), # four algorithms
                                 NbRunEval=3, # number of iterations
                                 DataSplit=70, # used for internal data calibration
                                 VarImport=10, # num of bootstraps to determine var importance
                                 models.eval.meth = c('TSS','ROC','ACCURACY'), # performance metrics
                                 do.full.models=FALSE, # run using all training data
                                 rescal.all.models = T, # need for ensemble compatibility

myBiomodEMHyacinth <- BIOMOD_EnsembleModeling(modeling.output = myBiomodModelOutHyacinthBig,
                                  chosen.models = 'all',
                                  em.by = 'all',
                                  eval.metric = c('TSS'),
                                  eval.metric.quality.threshold = c(0.7),
                                  prob.mean = TRUE,
                                  prob.mean.weight = FALSE,
                                  prob.cv = FALSE,
                                  prob.ci = FALSE,
                                  prob.median = FALSE)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.