2
$\begingroup$

I have a dataset of 4712 records and 60+ features working on a binary classification problem. I already tried out all the feature selection approaches like filter, embedded and wrapper but am just curious to learn and try genetic algorithm for feature selection.

The reason for choosing genetic algorithm is because I guess it will just provide me the best model fit based on best features.

1) I understand it might take time but would you people help me know how can I do this in Python?

2) In addition, is genetic algorithm any different or better than all other feature selection approaches discussed above? What are its disadvantages?

Is there any python packages and tutorial available on how to use this?

I see tutorials but they all about the theory of genetic algorithm

Can you help me by sharing any tutorial or package for genetic algorithm?

post update

enter image description here

$\endgroup$
  • $\begingroup$ Would it be possible for you to share the data? I'm GA researcher and would like to run some tests ;) $\endgroup$ – Piotr Rarus - Reinstate Monica Jan 3 at 7:53
  • $\begingroup$ Wow. Good to know. But Unfortunately, I can't. it's confidential. It's a tabular EHR data is the only info that I can share. Apologies $\endgroup$ – The Great Jan 3 at 7:58
  • 2
    $\begingroup$ Code your solution as binary genome. Either feature goes in or it goes out. To evaluate fitness train a model using xgboost will little trees. By default it's 100 so it's not that much. Do RepeatedStratifiedKFold with 10 repetitions. Take average f1-score as fitness value. It'll be good-enough estimation of feature set. $\endgroup$ – Piotr Rarus - Reinstate Monica Jan 3 at 8:06
4
$\begingroup$

Feature selection is a combinatorial optimization problem. And genetic algorithms is an optimization technique.

So there really isn't anything special, you just need to formulate your problem as an optimization one, and understand how do genetic algorithms optimize. There are enough tutorials on this.

Whether it's better or worse you already know the answer. It depends. On the dataset, constraints etc. What I can tell you from experience is that

  • You can not expect it to blow your mind but they do work pretty well
  • They are a great ensembler, meaning results are pretty different (yet accurate) from tree-based methods, NN etc...

Finally regarding implementation, here is completely (maybe too much) automated library based on genetic programming. (notice the word programming here referring to optimization not writing code) Also, it covers feature selection.

$\endgroup$
  • $\begingroup$ Hi @Noah, just a quick question - So in genetic algorithm implementation of Iris dataset, I see no feature being specified explicity. So the algorithm by itself will select the best features and return the best output. Right? $\endgroup$ – The Great Jan 3 at 8:21
  • $\begingroup$ yes, a bit too much as I said. And it runs pretty long also. $\endgroup$ – Noah Weber Jan 3 at 8:22
  • $\begingroup$ How can I know the best features? Will I get to know the feature names? $\endgroup$ – The Great Jan 3 at 8:22
  • 1
    $\begingroup$ I am sure there is an argument in the object thats returned giving you this possibility. Just check the docs $\endgroup$ – Noah Weber Jan 3 at 8:23
  • $\begingroup$ Thanks for your response $\endgroup$ – The Great Jan 3 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.