1
$\begingroup$

Context

When you receive messages in group chats, how do you detect if that message belongs to conversational dialogue or if it is a 'news' article (could be fake or real) that they are sharing?

Examples

Conversational dialogue: "Does anyone wanna have dinner today? I am pretty free this evening and I don't have dinner at home. Please let me know by 8pm!!"

'News' article: "New Japan’s growth has been helped by YouTube, which has made New Japan’s matches more accessible to an audience outside of Asia, said Dave Meltzer, publisher of the Wrestling Observer Newsletter, which has followed the sport since 1983. Capitalizing on this rise, New Japan launched an online streaming service — similar to the W.W.E. Network — in December 2014."

Question

Would you use rule-based matching or a classifier for this problem? (Assuming you already have a classifier for detecting if the news article is fake or real)

$\endgroup$
1
$\begingroup$
  1. News sentences will have more unique tokens than normal conversations.
  2. Conversations have more stop words than news articles.

I think you can use bert or normal wordvect classification to train a baseline model here. I would play aroud the pipeline of fake news classifier and news-conversation classifier. like passing the text to news classifier first and then passing it to news-conversation classifier. Try to mix and match to get the best results. Set some thresholds.

$\endgroup$
0
$\begingroup$

Rule base approach is always good starting point for initially as you might not have good amount of records to build classifier but be ready for certain false positive results.

Let's understand the false positive situation in case of rule base. In your conversational dialogue you have put keywords like dinner, home, evening. But in certain news case like Trump having dinner with CEO's in the evening inside white house this might categories into conversational dialogue class.

Now let's talk about classifier base approach. As you mention that there is already classifier for detecting fake news. You can use same classifier to decide whether its news or dialogue by setting up threshold based on accuracy let's say your model accuracy is 80%.

Now when you fake/Not fake news classifier predict something but the accuracy/probability is below 80% then assume it's conversational dialogue.

If you have good amount of datasets then always try to build your another classifier using pre trained embedding techniques to get good accuracy at early point.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.