# StackOverflow Tags Predictor...Suggest an Machine Learning Approach please?

I am trying to predict tags for stackoverflow questions and I am not able to decide which Machine Learning algorithm will be a correct approach for this.

Input: As a dataset I have mined stackoverflow questions, I have tokenized the data set and removed stopwords and punctuation from this data.

Things i have tried:

1. TF-IDF
2. Trained Naive Bayes on the dataset and then gave user defined input to predict tags, but its not working correctly
3. Linear SVM

Which ML algorithm I should use Supervised or Unsupervised? If possible please, suggest a correct ML approach from the scratch. PS: I have the list of all tags present on StackOverflow so, will this help in anyway? Thanks

• So the input is a body of a question and output is a list of suggested tags, correct? Sep 1 '15 at 10:52
• input= Training Data Set Questions output= Predicted Tags from Testing Set Sep 1 '15 at 17:17

This exact problem was a kaggle competition sponsored by Facebook. The particular forum thread of interest for you is the one where many of the top competitors explained their methodology, this should provide you with more information than you were probably looking for: https://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/forums/t/6650/share-your-approach

In general, it appears that most people treated the problem as a supervised one. Their primary feature was a tf-idf, or unweighted BOW, representations of the text and they ensembled 1000s of single-tag models. Owen, the winner of the competition, noted that the title text was a more powerful feature than the content of the body of the post.

One interesting algorithm that I've once tested is called TopMine: http://web.engr.illinois.edu/~elkishk2/ (under Code and Datasets). It is able to extract bi-grams that could be used as key words and it can also assign them into topics.

If you have the tags of each of the questions that you mined, then supervised methods make sense.

You might use the tf-idf representation of a given question, feed it to an SVM or neural net, and use that to predict 0/1 for each tag in your target set. If there are too many possible classes (tags), it may be tricky to balance your data.

A simpler approach might be to use the tf-idf vectors to compute the K-Nearest-Neighbors of a question; then you can use the tags of the most similar documents (by whatever distance metric does best) to predict the likelihood that question has each tag.

If you don't have the tags for the questions you mined, you should consider unsupervised methods. LDA, for example, could identify topics within the questions and important words within those topics

• I have the tags with the mined data. 1. is given question mentioned by you, the question from which tags are to be predicted or from the training set..please clear out what is target set? 2. What are TF-IDF Vectors? Sep 1 '15 at 16:38
• by the target set, I simply mean the values your output could take on. If you have N possible tags, then your output could be a N-dimensional vector with 1's for the index of the relevant tags and zeros elsewhere. Let's say there were only 4 tags you cared about predicting: "machine-learning," "networks," "classification," and "hardware." Your output vector for this question would look like: var | machine-learning networks classification hardware q | 1 0 1 0  Sep 1 '15 at 17:24
• When you use tf-idf (term frequency, inverse document frequency), you have a 'vocabulary' of terms. Each document is represented by a vector. Each index of the vector is a term and its value is the term's frequency divided by the number of documents that term shows up in. There are other ways to calculate tf-idf vectors as well shown here Sep 1 '15 at 17:31