1
$\begingroup$

I have a bunch of documents that I want to classify which ones talk about soccer (unsupervised learning, I do not want to manually label the documents).

One way I am thinking about is to go online and search for the most popular words in soccer articles to make a list of vocabularies (for example: score, shoot, World Cup, etc). Then somehow use that list of vocabularies to classify the documents (maybe if a particular contains 30% of the words in that list of vocabularies, then that document talks about soccer).

I am wondering whether it is a valid method or there are better existing methods. Really appreciate any help.

$\endgroup$
0
$\begingroup$

First of all you need to have available a train set, which means that you should annotate manually which document is related to soccer and which not. Then you need to process the available corpus (remove numbers, stop-words etc., stemming) and build a vocabulary. After that you should choose the appropriate feature representation. Each term is a feature and you have to decide how you are going to reprsent each feature, which means what kind of weight you will assingn. One way is the tf-idf representation. Then you will be able to train a classifier.

*The only way to avoid labeling manually the texts is to find some already labeled in the same language.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks, I forgot to include in the question I do not want to manually label the documents. $\endgroup$ – Tuan Do May 29 '19 at 17:44
  • 1
    $\begingroup$ Then it is not easy to build a classifier without labeled data. If you want to consider some key words only as you mentioned then this is more similar to information retrieval task where you can measure the similarity of the vectors using a metric like cosine similarity. $\endgroup$ – Christos Karatsalos May 29 '19 at 21:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.