I have a data set of questions belonging to ten different categories namely (definitions, factoids, abbreviations, fill in the blanks, verbs, numerals, dates, puzzle, etymology and category relation).
The categories are briefly described as follows:
- Definition – A question that contains a definition of the answer.
- Category Relation – The answer has a semantic relation to the question where the relation is specified in the category.
- FITB – These are generic fill in the blank questions – some of them ask for the completion of a phrase.
- Abbreviation – The answer is an expansion of an abbreviation in the question.
- Puzzle – These require derivation or synthesis for the answer.
- Etymology – The answer is an English word derived from a foreign word.
- Verb – The answer is a verb.
- Number – The answer is a numeral.
- Date – The question asks for a date or a year.
- Factoid – A question is a factoid if its answer can be found on Wikipedia.
I used the Stanford core NLP package called shiftreducer to find out the Part-Of-Speech (POS) values for each question in a category. I thought of using this POS pattern as a discriminant among the classes but it turned out to be generalized since:
- All the classes follow a similar pattern
- Nouns top the POS count followed by Determinants, Prepositions, Adjectives, Plural nouns and finally verbs.
What could be the other ways in which I could differentiate among the question categories? Or as my question was in its first place, "What kind of features do I select for efficient categorization?"