Is this a good approach to classify tickets which contains description and logs?

Question

I want to classify a dataset of support tickets which mostly contain text in the description field and sometimes server logs in a separate field.

The log field is not always there but when it's present, it's a good indicator of the target class of the ticket.

I have created a CNN based classifier which can classify the tickets based on the log field, and a SVM clf with TFIDF based features for the description field.

I am thinking of adding the output probabilities of the CNN classifier in TFIDF based SVM classifier to combine the models as a feature column.

Is there a better way to combine these models?

Is there a better way to approach this problem, without having two separate models?

Answer 1

First, you should get clear about your non-functional requirements (speed, memory consumption). Make sure you have a clear idea who is going to maintain the software and what the effort will be if you make some changes. Higher accuracy might not be worth it.

The simplest way to combine n models is averaging their predictions. This technique is often used on Kaggle. See chapter 2.6 of https://arxiv.org/pdf/1707.09725

Did you try a simple MLP with tf-idf features? How well does that perform? Usually it executes way faster than SVMs, if you have many classes.

Answer 2

Read about Stacking. In layman's terms terms you can build both the models and output of these two can be features of another model. Hope this helps.