Training a classifier with text and numerical features - what is the state of the art?

I'm trying to build a binary classifier where the features are mostly numerical (about 20) and there are a couple of unstructured short text fields as well. What is currently considered the state of the art for combining these types of features?

I've tried building a separate classifier (logistic regression, TFIDF) using the text data alone, and then including that output score of that classifier as an additional when training using the rest of the numerical features (random forest, taking care to train each classifier on different folds of the data to prevent signal leakage). It works alright, but I think a better performance is possible.

A variant is to simply train two separate classifiers, one using text and the other using numerican features and then stacking those two. And finally, another idea is to use neural networks, have two input networks, one CNN/LSTM for the text, and another dense for the numerical features, and then combining them and having a single output.

Are there other approaches I haven't thought of that are worth a try?

• I don't want to answer you yes or no, but I tried this way and it works well. I tested it in my approach for a research shared task and I was ranked first. Despite that, what you've mentioned is still a scientific explanation. BTW, I do this with classical classifiers (SVM, NB, etc.), in the case of Neural Networks, I concatenate two different branches (one for text and another for the numerical values) at the very last steps of the network. – Ghanem Apr 22 at 17:14