1
$\begingroup$

I am in an ML course and one of our tasks is to predict the helpfulness of Amazon reviews. Currently, I am doing what most people seem to . That is, a hashvectorizer(2-gram) on the text, tfidf, several scalar features, and (in my case) a Ridge classifier (parameters chosen by grid search). Honestly, it has been frustrating since I have tried to read and apply best practice from the course and any published works I can find. That said, I am still only achieving marginal results:

{'Pos': 6588, 'Neg': 84412, 'TP': 2755, 'TN': 78990, 'FP': 5422, 'FN': 3833, 'Accuracy': 0.8982967032967033, 'Precision': 0.33692063103827807, 'Recall': 0.41818457802064357}

...which isn't great. I don't need help coding*, but if anyone can suggest useful case studies, or other sklearn classifiers/techniques I could research, I would be grateful. I'm rapidly running out of directions to follow up on.

Thanks!

  • if relevant, you can find my latest iteration here
$\endgroup$
1
$\begingroup$

Naive Bayes is almost always my first go to for a problem with text data. This is primarily due to the independent distributions that integrate so well with the document by feature independent style matrix by nature. I would extend your idea of Ridge with Lasso or ElasticNet. I would not recommend normalize your data if it is sparse, this has been known to cause problems, but StandardScaler is a rather conventional way to get a centered dataset without producing disharmony among the variables in the data. I would consider your error metric of precision or fmeasure under each of the above models as my personal next step, then do some visualizations to argue your best fit.

$\endgroup$
0
$\begingroup$

If you want to stick with vanilla Machine Learning, SVMs hav been known to work really well on text classification problems. You can experiment with various kernels. Alternatively, you can try neural networks with word embeddings. However from your evaluation metrics, it seems you have a data imbalance problem, hence the low precision and recall. Accuracy becomes completely meaningless here. Before you try anything else, I would suggest that you try to balance your samples by getting more data, if that doesnt seem to be possible then try bagging.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.