How to use multiple text features for NLP classifier?

Question

I am trying to build text classifier, Usually, we have one text column and ground truth. But I am working on a problem where dataset contains many text features. I am exploring different ways how to utilize different text features.

For example, my dataset looks like this

Index_no                   domain  comment_by   comment       research_paper      books_name

01                         Science  Professor   Thesis needs  Evolution of         MOIRCS 
                                                more work     Quiescent            Deep 
                                                              Galaxies as a        Survey
                                                              Function of
                                                              Stellar Mass       



02                         Math    Professor   Doesn't follow  Evolution of   
                                               Latex format   Quiescent           nonlinear 
                                                              Galaxies as a       dispersive
                                                              Function of         equations
                                                              Stellar Mass             

This is just a dummy dataset, Here my ground truth (Y) is domain and features are comment_by, comment, research_paper, books_name

If I am using any NLP model (RNN-LSTM, Transformers etc), those models usually take one 3 dim vectors, for that if I am using one text column that works but How to many text features for text classifier?

What I've tried :

1) Joining all column and making a long string

Professor Thesis needs more work Evolution of Quiescent Galaxies as a Function of Stellar Mass MOIRCS Deep Survey  

2) Using a token between columns

<CB> Professor <C> Thesis needs more work <R> Evolution of Quiescent Galaxies as a Function of Stellar Mass <B> MOIRCS Deep Survey 

where <CB> comment_by , <C> comment, <R> research_paper, <B> books_name

Should I use <CB> at the beginning or use like this?

Professor <1> Thesis needs more work <2> Evolution of Quiescent Galaxies as a Function of Stellar Mass <3> MOIRCS Deep Survey

3) Using different dense layers (or embedding) for each column and concatenate them.

I've tried all three approaches, Is there any other approach I can try to improve the model accuracy? or extract, combine, join the better features?

Thanks in advance!

Answer 1

One option is to embed all the information in a single space. The embedding space would contain the tokens and feature names.

Often times the tokens are changed to track the provenance. For example, science__DOMAIN and professor__COMMENT_BY.

An example of a package that does that is StarSpace.