1
$\begingroup$

I have a large dataset (195 features x 20m samples) that I have trained using XGBoost. I would like to see if a genetic algorithm can beat XGBoost since the data has so much noise it is prone to overfitting.

I would like to use a tree-based model so I don't have to standardize the data, and the features do have some interrelationships.

Are there any python packages that have this all done? Ie.that can create trees through a genetic optimization process?

$\endgroup$
1
  • $\begingroup$ With the correct regularization + CV in XGBoost you should not have overfitting problems $\endgroup$ – Julio Jesus Apr 7 at 17:16
2
$\begingroup$

Very new release:

genetic-tree


The main objective of the package is to allow creating decision trees that are better in some aspects than trees made by greedy algorithms. The creation of trees is made by genetic algorithm. In order to achive as fast as possible evolution of trees the most time consuming components are wrtitten in Cython. Also there are implemented mechanisms for using old trees to create new ones without need to classify all observations from beggining (currently in developmnet). There is planned to allow multithreading evolution. The created trees should have smaller sizes with comparable accuracy to the trees made by greedy algorithms.

Also worth to be checked:

PyGAD


PyGAD is an open-source Python library for building the genetic algorithm and optimizing machine learning algorithms.

References:

https://hal.inria.fr/hal-01405549/document

https://www.hindawi.com/journals/tswj/2014/468324/

https://www.kdnuggets.com/2018/07/genetic-algorithm-implementation-python.html

$\endgroup$
1
  • $\begingroup$ Thank you for mentioning PyGAD :) $\endgroup$ – Ahmed Gad May 8 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.