There is some important information missing in your question, i.e. what the standard parameters are and what kind of logistic regression you use.
When you use sklearn.linear_model.LogisticRegression
, you will see in the docs that the first hyperparameter is the penalty
which defaults to l2
. This means that by default "shrinkage" of parameters is used. By using regularization, features which are not very helpful in predicting some outcome are "shrunken". This is exactly what you will do when you have "high dimensional" data (lot of features and not so many observations).
Tree based boosted models are not per se "bad" in high dimensional settings. However, in order to achieve good performance, it may be neccesary to introduce column subsampling and probably row subsampling ("stochastic gradient boosting") as well as "enough" boosting rounds (probably with a low learning rate). The reason for this is that you want to show the boosting algorithm as much information as possible while avoiding dominance of few "powerful" features. Since in each boosting round only "shallow" trees are grown (usually 5-8 splits), few "powerful" features will dominate in case you do not randomly sample columns (features) in each boosting round. Thus, important details may not be learned when few powerful features dominate.
You could inspect the feature importance from both models to see if there are strong differences.