I am working on my first project, I am trying to predict the quality of a software specification requirements. I have 1000 requirements which have been manually labelled on a scale of 1-5 (poor-excellent) quality. My idea is to use doc2vec to in order to predict the quality of a requirement.
I have experimented by following this example: https://towardsdatascience.com/implementing-multi-class-text-classification-with-doc2vec-df7c3812824d
I am not sure if this is a viable solution as the results are not very good, but I'm not sure if this is due to my data or the model.
Each requirement ranges between 1 to 4 sentences, I use taggedDocuments to label the requirements as either 1, 2, 3, 4, 5. Would the correct way to go about this be to label the data according to the 5 classes or should I be labelling them based on each individual requirements.
I have read these answers;
however, I'm still not sure what to do in my particular circumstance so any advice would be greatly appreciated!
Also, I am assuming that word2vec would not be of use to me in this instance as I am interesting in associating similar requirements not the actual words...