I'm doing text classification with SVM. I'm using Tfidf vectorization. In addition to the text vectors, I have a context data denoting the possible outcomes of the prediction. For example, I have a text about 'Mona Lisa' which should be classified as 'Art'. My context vector includes possible outcomes for this text such as ['Art', 'History', 'Culture']. These vectors are multi-hot encoded, concatenated with text vectors and fed to the algorithm. I have 0.72 accuracy without context vectors and 0.78 accuracy with them. So it works. But I think it might be better if I can boost the effect of the context vectors to the outcome.
Each of my text vectors size is 10k. Context vectors' size on the other hand is only 80. So I have a 10080 sized vector as input to the SVM. What I'm asking is if is there any way to make these 80 sized context vectors be more discriminative in the algorithm?
I'm open to suggestion for a deep learning architecture as well. I tried with DNNs and had improved results with context vectors - though not as good as SVM.