1
$\begingroup$

I have a small data set (4000 records with 10 features) and I used XGBOOST in R as well as Boosted Decision Tree model in Azure ML studio. Unfortunately the results are different. I like to optimize recall and I could pick that as a measure in Azure but I can not do so in R.

I used the same parameters in both platforms. I know seeds might be different but I tried many of them. I always have a much better recall on my validation dataset using the Azure model compared to the R one.

I wonder if there is a big difference behind the methodology used in these two platforms causing me the issues.

I also used cross validation which did not help. Any insight is appreciated.

Thanks

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

It's hard to say, without being able to know exactly what Azure is doing.

  • From what they do share, they bin continuous features; you could try tree_method='hist' in xgb to be more similar there.
  • I can't tell how Azure deals with categoricals or missing values.
  • Be sure to set xgb's max_depth=0 and grow_policy='lossguide', since you want to use max_leaves instead for a direct comparison.

https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/two-class-boosted-decision-tree#usage-tips
https://docs.microsoft.com/en-us/azure/machine-learning/studio-module-reference/two-class-boosted-decision-tree#module-parameters

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you Ben. I did that however the model gives me the same prediction for all of my validation data. If I remove the max_depth=0 , things are back to how they were before but when I add that to the function, it does not work properly. $\endgroup$
    – Fatima
    Sep 30, 2019 at 18:12
  • $\begingroup$ @Fatima, It works for me. Can you provide your code? $\endgroup$
    – Ben Reiniger ♦
    Sep 30, 2019 at 18:25
  • $\begingroup$ Here is the code that I have: bst <- xgboost(data = as.matrix(Train %>% select(-spam)), label = Train$spam, max_depth =0,max_leaves=100,grow_policy='lossguide', eta = 0.1, nrounds = 350, objective = "binary:logistic") $\endgroup$
    – Fatima
    Sep 30, 2019 at 18:46
  • $\begingroup$ @Fatima Needs tree_method='hist', I think. $\endgroup$
    – Ben Reiniger ♦
    Sep 30, 2019 at 18:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.