0
$\begingroup$

For text classification with machine learning - If your training data was generated purely with regular expressions, is it possible to train a machine learning model with this training data which will be more accurate than your original regex?

  • By 'accuracy' I mean to say the percentage that are correctly labelled as compared to the human truth of the question.

(E.g. we have a set of Tweets that we've labelled as happy or sad with regex. I'm wondering if a machine learning classifier trained on this data will be able to surpass the accuracy of the regex that generated the training data)

$\endgroup$
1
$\begingroup$

No- not if your regex is actually generating the data. If this is the case, the regex (or a complementary one) should be able to classify the resulting text perfectly. That is, you know what the rules for creating a 'happy' or 'sad' piece of text are within the regex, and so you simply apply those rules backwards to classify it.

So, if you take what the regex says when it creates the data as ground truth, then there's no reason to create a machine learning classifier to classify it- the creation mechanism itself already does so.

On the other hand, if you have created a regex that will produce "happy" or "sad" tweets, and you want to know if the regex itself or a machine learning algorithm will better conform to the opinions of, say, human observers, you would have a very interesting question to evaluate and a relatively straightforward test case for sentiment analysis.

$\endgroup$
0
$\begingroup$

It depends. :-) Depends on many factors but first of all on your data of course. As a corner case let's take a clustering algorithm on the perfectly separable data. Theoretically it could then reach 100% precision even without using your regex generated training data. You need to limit your question to include specific dataset characteristics to have an actionable answer.

$\endgroup$
0
$\begingroup$

I was looking for something similar and found those https://www.import.io/post/neural-nets-how-regular-expressions-brought-about-deep-learning/ and this tool https://regexpgen.import.io/

I hope this would help

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.