Using a combination of gradient boosting with LSTM for classification?

Question

I am presently using an LSTM model to classify high dimensional tabular data which is not text/images (dimensions 21392x1970). I also tried XGBoost (Gradient boosting) in Python separately for the same classification task (classify into one of 14 categories of different categorical values). I have come across the provision of using feature_selection_ method in XGBoost, which can provide me the F1 scores of the most relevant features in prediction.

I would like to create a hybrid model that combines the LSTM with the XGBoost but am confused as to how I can do something such as using the most important features for classification (probably getting these through XGBoost) and then feeding to an LSTM ?)by a combined approach. Any ideas, suggestions, and comments are appreciated!

Answer 1

I'm not aware of any streamlined way of doing this without writing the proper code to connect the two models. It should be fairly straight forward to run XGBoost, grab the feature importance and use this as a way to identify the features to use for the LSTM. This part should be not so hard, just have a list of features with some criterion of whether to keep or drop it. Once you have the list of features you want to use for the LSTM, just filter the whole dataset, and build out the LSTM model using that set.