I am a newbie to data science. I have a 'short text' categorization problem where input variables are either unstructured texts (names, definition, description etc) or categorical. There is not much semantic to the fields as they are product names, territory name, sales order type etc. Issue is I do not have any sample data set from which I can derive training, test, validation set or divided it into k-fold for cross validation. So how should I generate sample data? I have about 20 target classes. I can classify some dataset using regex or lucene rule based matches and manually verify them and make sure each class have equal amount of samples. But I am open to other suggestion.

  • You would like sentence wise classification problems? – Hima Varsha Jul 18 '16 at 6:44
  • @HimaVarsha Not sure what you mean? do you mean semantic analysis of text? – nir Jul 18 '16 at 19:03
up vote 5 down vote accepted

I know this isn't answering the question that you actually asked, but I suggest that you NOT generate data for your 'short text' categorization problem.

Generated data can work for certain cases when data scientists who are very familiar with an algorithm want to demonstrate a specific feature, but there is a hokeyness that may lead you astray as someone new to data science and machine learning.

I suggest you avail yourself of some free and open data here, or an entire stack exchange dedicated to open data.

Real data will give you experience with less contrived problems and you can often try out similar ML models on very different data sets in order to gain experience with the variations and pitfalls of data science. Be creative... you might not find exactly what you want, but you can subset the data by projecting out a few text fields and using them to classify another field.

Addendum update:

With your recent edit, it is now apparent that you are seeking to turn your unsupervised training data into supervised training data in order to train a supervised learning classification model. The method that you suggested, "classify some dataset using regex or lucene rule based matches and manually verify them", is a deterministic unsupervised method without the human verification step.

Without the human verification of the target data that you have created, I would not consider feeding the derived targets back into the classification algortihm as your results will only be as strong as the derived target data and predictions will show similar errors. Instead, you should think about a semi-supervised learning method where you perhaps employ a clustering algorithm and then label the clusters with the target variable.

With human verification of the target data that you have created, this will work just fine as training data for a classification model. The only issue is that this can become tedious. There are mechanical turks (humans paid to perform repetitive tasks) that you can hire to perhaps perform the labeling for you, which may be a more scalable option.

Hope this helps!

  • Do you mean I can train model on publicly available dataset which does categorization of short text and apply that to my dataset! How that will work since I have different classes. Or are you suggesting I should try find similar data set and classes for categorization? It's a smart idea though in general. – nir Jul 18 '16 at 19:00
  • In editing the question you have added new information and changed the scope of the question. I now see that you are seeking to turn your unsupervised training data into supervised training data by determining the target variable and then attempting to feed this back into your supervised learning model. I will add an adendumm to address this. – AN6U5 Jul 18 '16 at 19:53
  • I am also mentioning that I'm newbie :) What do you mean I have 'unsupervised training data' ? Do you mean my problem naturally falls in to unsupervised learning and I'm trying to force it to supervised learning? – nir Jul 18 '16 at 19:58
  • Yeah. There's no problem with adding target data that is hand created, but scalability becomes an issue. Mechanical turks can help, but this can cost a bit of money. Semi-supervised learning may be the solution you are after. – AN6U5 Jul 18 '16 at 20:04

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.