What does many low important feature indicate?

Question

I have a dataset where I am focusing on binary classification problem. In total,I have around 60 features in my dataset

When I used Xgboost Feature Importance, I was able to see that the top 5 features account for 42% whereas rest of the of the 50 features account for 40-49 % (each feature about 1%) and remaing 8-10 features have zero importance or less than 1% of importance.

This is my best paramter list for Xgboost after gridsearch

op_params = {'alpha': [10],
 'as_pandas': [True],
 'colsample_bytree': [0.5],
 'early_stopping_rounds': [100],
 'learning_rate': [0.04],
 'max_depth': [6],
 'metrics': ['auc'],
 'num_boost_round': [10000],
 'objective': ['reg:logistic'],
 'scale_pos_weight': [3.08],
 'seed': [123],
 'subsample': [0.75]}

Since I have many low importance features, should I try to use them all in my model to increase the model metrics?

When I built the model only with top 5 features, I was able to get 80% accuracy.

I am trying to understand is it even useful to make use of these low importance feature for prediction?

Shown below is my feature importance in descending order

Do they even really help?

Any insights would really be helpful

Answer 1

Its all about a tradeoff.

The more you add unimportant features, the marginal will the benefits get but you risk injecting more complexity and potentially overfitting.

Ocams Razor

Also be carefull with the default feature importance approach. Read this.

Answer 2

Adding low-value features might not help you surpass your current accuracy. Getting good quality data and add more data to the dataset or training for more epochs if it doesn't converge might help you get more accuracy.