5
$\begingroup$

Currently I am using LDA to apply topic modeling to a corpus. Since LDA is unsupervised, it returns a set of words for a given 'topic' but doesn't necessarily specify the topic itself. I was wondering if there are any suggestions for algorithms that take a list of words and sees what topics it can be categorized to?

For example [cat, dog, fish] can be categorized to animals or pets.

One output for my model: 

['game', 'week', 'fantasy', 'sportsline', 'play', 'players', 'league', 'random', 'sunday', 'season', 'agent', 'elink', 'exercise', 'start', 'yards', 'free', 'injury', 'expected', 'practice', 'getbad', 'weekly', 'year', 'reports', 'starting', 'luck', 'nat', 'nfl', 'weeks', 'smith', 'fast']

Could be categorized to football or sports.

Any suggestions, specifically with Python models/packages would be much appreciated.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ You stated "Could be categorized to football or sports" which been found based off n your knowledge that been developed by your experience. To expect the same from a machine, you need to help it experience this relationship between the output of your model and your finding. $\endgroup$
    – krayyem
    Nov 10, 2018 at 8:00

1 Answer 1

Reset to default
0
$\begingroup$

Extending the basic idea stated in comments by @krayyem, you are looking for an Ontology and/or Taxonomy. The short story is to (1) build them based on your data or (2) use existing ones.

Building from your data

You need lots of Is-A pairs which indicate what kind of thing the concept is e.g. Maradona is a football-player and football is a sport. For this purpose, you extract info from your text based on some patterns and update info then update pattern based on info and the loop goes on till it does not change anymore. see this answer though.

Solution you are probably looking for

Use existing knowledge bases that some of which are in Python e.g. wordnet. You can find more in the links I provided for the answer I linked above.

How to proceed kind of idea

Find the dominant keyword in the found keywords. You may simply just count and go through the concept graph (your ontology, the knowledge base) and find the parent concept. If the graph is not accessible in this form, you may create it yourself in Python with existing APIs and/or graph libraries in Python like Networkx.

If more input is needed please comment here. Good Luck!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.