I tried exactly same dataset in Orange 3 and Azure ML Studio for linear regression. Coefficient of determination in Azure ML studio is .71 while Orange gives R square as .45. Why such a huge difference?
1 Answer
It could help if you have added screenshots of both models settings. However, what I can say is Orange and Azure ML Studio use different parameter settings. It might be possible that some are available in one of them but not in the other.
If we assume that you have exactly the same configuration in your two models, then the random parameter can also play a big role here. Even if you use the same seed, Azure ML and Orange, probably, don't use the same random generator algorithm. As a result, the outcome of your models is not replicable.
I would advise trying with cross-validation in both implementations and dig more on the evaluation metrics. If you need more help from the DS community, you might need to add more details as well.