0
$\begingroup$

I have a problem. There is a dataset A, which deals with a classification problem. And for this dataset, several different baseline algorithms have been defined and computed.

In addition, three models were used: Logistic Regression, XGBoost and RandomForest.

Now my question is this, why use different algorithms (Logistic Regression, XGBoost and RandomForest) and investigate which one is the better algorithm? Is it because the algorithms have different strengths and perform better depending on the data set?

Algorithm                               Accuracy Precision Recall F1-Score
Baseline 1                              0,20      0,20     0,20   0,20
Baseline 2                              0,20      0,20     0,20   0,20
Logistic Regression                     0,53      0,52     0,28   0,36
RandomForest                            0,65      0,64     0,63   0,63
XGBoost                                 0,50      0,61     0,55   0,58

For example, RandomForest, gave the best result and then the hyperparameters are adjusted.

$\endgroup$

1 Answer 1

2
$\begingroup$

In machine learning there is something called the no free lunch Theorem, which basically states that there isn't one solution/algorithm that will perform best on every problem. Different algorithms will perform differently on different data. Therefore, you try different algorithms to pick the best, although there are algorithms that are generally more powerful for certain types of problems (...).

Hint: You usually pick the best algorithms without hyperparameter tuning in the first round and then compare the best ones again after tuning their hyperparameters since these can make a decisive performance difference - especially for more "complex" algorithms like XGBoost in contrast to Random Forest. Although both algorithms are decision-tree based.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.