Detecting low-quality, user-created text content

Question

I run a website that allows visitors to publish text content. It is very similar to a forum in terms of functionality. I'd like to automatically exclude or flag, submitted text content that is "spammy" or "low quality" (subjective I know). Obviously I want to detect content that is offensive or classic spam. I also want to detect content that is meaningless or nonsense. I know this must be a common problem for those that run web apps that allow user-submitted content, but I haven't found an obvious solution yet.

Ideally I'd like to use open source software that can run on Linux, suitable for me to install on my webservers, or an AWS service.

I've tried AWS Comprehend which, without any custom training, can detect the sentiment of text accurately, It doesn't seem to detect low-quality or spam out-of-the box. It supports training of custom classifiers, you simply provide a csv containing class/document pairs. This could be a good way to go, can anyone suggest a free dataset that I could use to train it?

It seems like text quality detection would be such a common requirement that I thought there might be some software out there that is pre-trained for that exact purpose, perhaps a Linux package. Does anyone have any suggestions?

I wasn't sure wether to post this in Data Science or Programming, I apologize if this is misplaced.

Answer 1

There might not be a off-the-shelf package for detecting "spammy" or "low quality" because is it is relative to the context (as you already recognized).

One option is to use rules-based approach. Have a list of spam keywords. If any of them appear then stop the text. This is simple but also a limited approach.

Another option is to use machine learning learning to train a system to detect spam. The first step is collect thousands of human-labeled examples of both spam and not spam. Then apply natural language processing text classifier, naive Bayes classifier is a good baseline.