1
$\begingroup$

I'm trying to optimize my features in a dataset to get a better predictive model. I used Exhaustive feature selector from mlxtend. This checks all possible combinations of features. I have a dataset with 80+ features, and I set the maximum of features to be selected to 20 with 10-Fold cross validation. There are 9k entries and the estimator is Random Forest Regressor. It's been roughly three days this has been running. Is this normal? Previously I have run Recursive Feature Elimination on the same dataset with 10 fold cross validation. It took roughly 24 hours.

efs = EFS(RandomForestRegressor(), 
      min_features=3,
      max_features=20,
      scoring='neg_mean_squared_error',
      cv=10)

     efs.fit(X, Y)
$\endgroup$
2
  • 1
    $\begingroup$ yes given the size of dataset and combinations it can be normal $\endgroup$
    – Nikos M.
    Dec 4, 2021 at 19:37
  • 1
    $\begingroup$ I had one which ran for a week. You could reduce cv to save some time. $\endgroup$
    – Peter
    Dec 4, 2021 at 21:35

1 Answer 1

3
$\begingroup$

An exhaustive search means that the 10-fold CV is run for every possible subset of features. The number of possible subsets of size 20 for a set of 80 features is:

$$\binom{80}{20}=3.5\ 10^{18}$$.

And this is only for size 20 exactly, so the total number is higher.

So yes, it might take a very long time. You could estimate the duration before running a potentially very long experiment like this:

  • First take a lower maximum number of features and calculate the corresponding number of runs. For instance with 5 as maximum the number of subsets would be roughly around 24 millions (I let you calculate the exact number).
  • Then run the experiment and measure the duration. Applying a simple proportion you can estimate how much time it would takes for any maximum number of features. You might find out that your current experiment would take many years ;)

Exhaustive search on a large set is rarely a good idea, I'd suggest looking at genetic search.

$\endgroup$
2
  • $\begingroup$ I will do for maximum features of 5, and with no cross validation. Will post how long it took. Would you suggest any particular GA’s and where are they implemented? $\endgroup$ Dec 5, 2021 at 17:42
  • 1
    $\begingroup$ @PlatinumMaths I haven't used it myself but there is this library for python. $\endgroup$
    – Erwan
    Dec 5, 2021 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.