1
$\begingroup$

I am currently working on a project where my job is to intent analysis of an article. Suppose i am given with the article and i want to classify that what kind of article it is?. Any suggestions, Which kind of model would be best to apply?

$\endgroup$
1
  • $\begingroup$ Find embeddings of the query and the documents, then perform similarity search. Use pre-trained embeddings, if possible. Welcome to the site! $\endgroup$
    – Emre
    Nov 28 '17 at 18:42
2
$\begingroup$

I think the easiest way would be to cluster them with wordvectors. The "20 Newspaper" dataset would be a good test for the algorithm http://qwone.com/~jason/20Newsgroups/

On the Sklearn Website, you will find some easy example for using wordvectors and classification: http://scikit-learn.org/stable/datasets/twenty_newsgroups.html

If you go deeper into this space you could start using LSTM networks and neural networks, but this is a little bit harder to develop. So first, try to use the problem statistically. If you want to analyze the text semantically you will need LSTM.

I hope I could help you.

$\endgroup$
2
  • $\begingroup$ Hi Christian thanks for your answer but i am looking more like the keyword based model. for example if you i give some particular keywords , my model look from set of of some articles and give the best articles(keyphrase extraction) $\endgroup$ Nov 28 '17 at 16:14
  • 1
    $\begingroup$ @kanavanand: If you know more about what you are looking for, please put that into the question. You asked for "any suggestions" with no details other than you want to classify articles. Your comment here implies you don't even actually want to classify the articles, but find similar articles in your dataset. Please have a think about what it is you really want and edit your question to include that information. This is important, because writing answers takes time and effort, no-one wants to do that if it isn't even useful to the person asking. $\endgroup$ Nov 28 '17 at 20:04
2
$\begingroup$

It sounds like you are trying to classify each article based on a set of tags that are inherently generated from each of the articles.

As an alternative to Christian's solution (which I think does serve the purpose that you had described), you could consider using an N-Gram model and Parts-of-Speech (POS) tagging in order to do the following (in this order):

  1. Calculate the frequency of recurring, common phrases within a document / corpus
  2. Apply POS tagging to classify nouns (proper and common), verbs, adjectives... etc.
  3. (Optional) Apply some sort of rule to only select the top Y most frequent terms

Using a standard language model, you can compute the relative probabilities of those words occurring within each document to then weight which article is most likely to mention a certain tag.

You can learn about creating frequency tables, POS tagging, calculating relative frequencies with the attached links (in python).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.