8
$\begingroup$

I have 40000 rows of text data of health care domain. Data has one column for text (2-5 sentences) and one column for its category. I want to classify that into 300 categories. Some categories are independent while some are somewhat related. Distribution of data among categories is not uniform either i.e some of the categories(around 40 of them) have less data about 2-3 rows.

I am attaching log probablity of each class/categories. (OR distribution of classes) here. Class prior logarithm of probabilities (log class distribution of data)

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Need more information. What is the relationship among the categories? Are the categories mutually exclusive? Is there categorical overlap? $\endgroup$
    – Ryan J. Smith
    May 7, 2015 at 16:29
  • 3
    $\begingroup$ Welcome to Data Science! Currently your question is of very low quality. You can't expect quality answers without asking well described questions. Please, provide more information (better description of the data, of your background, programming languages, researched approaches etc.). $\endgroup$
    – Wojciech Walczak
    May 7, 2015 at 17:09

1 Answer 1

Reset to default
8
$\begingroup$

In general, a decent starting point for problems like these is Naive Bayes (NB) classification using a simple bag of words model. Here are some slides describing NB as applied to natural language processing. There's nothing especially fancy about this approach, but it's pretty easy to implement and will give you a starting point to expand from.

Once you've found some initial results assuming independence among your features and your output labels, you'll probably have a better sense of where the model is weak. From that point forward you can apply some feature engineering (maybe TF-IDF) as well as some post processing to deal with samples that get assigned to related categories.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I appreciate your answer and the references here, even if the question is vague. It's really helpful to me and probably a lot more people who are just getting their feet wet as well. Thanks! :) $\endgroup$
    – Brian Topping
    May 11, 2015 at 11:16
  • $\begingroup$ Thanks, I have started working with naive bayes and feature engineering in general. Any other things apart from naive bayes that I should try? $\endgroup$
    – Alok Nayak
    May 15, 2015 at 14:31
  • $\begingroup$ Well, you still haven't offered very many details about the data itself or the specifics of what you've done, so it's very difficult to give you specific suggestions. The best I can say is consider incorporating some sequential structure into your model and features either through use of bigrams or markov models / finite state machines. $\endgroup$
    – Ryan J. Smith
    May 15, 2015 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.