2
$\begingroup$

I have a data set of tweets regarding vaccines. They have been collected from an API because they have keywords like "flu, measles, MMR, vaccine" etc.

I need to find tweets specifically about measles and the outbreak that occurred in California this past February. It isn't enough to search the data set for words like "California" and "Measles" because tweets like "MMR vaccination rates in Palo Alto on the rise" are about measles and California, but wont be captured by a naive search.

Are there any unsupervised algorithms that could help me out?

$\endgroup$
  • $\begingroup$ You could use a simple binary classifier. $\endgroup$ – Johnny000 May 14 '15 at 23:40
2
$\begingroup$

A couple ideas:

  1. If you have a large data set of tweets, you could try using latent semantic indexing to find out which terms are semantically related based on their usage and co-occurrence. Then after transformation, you can apply some document similarity metric, e.g. cosine similarity, to find tweets most relevant to your queries, e.g. "California measles".
  2. Use knowledge or lexical databases like DBPedia and WordNet to calculate semantic relatedness between your query and the text of your tweets, or to identify tweets with related concepts.
|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.