Orange Text Mining Data Format

Question

Is there a special header structure required for text classification problems?

I'm trying to do Tweet classification using the text mining plugin. I have the data in a CSV file, with one column labeled category and containing my two categories, and the other labeled text containing the tweets.

When I load the file with the corpus widget and follow the workflow given at http://orange3-text.readthedocs.io/en/latest/widgets/bagofwords.html the test widget gives the error "Train data input requires a class variable."

I opened the sample bookexcerpts.tab data and discovered it has a different header structure than just category and text. If I modify my tweet CSV to match it seems to work. Is this a bug, or am I missing a step?

Answer 1

No, no special structure is required for text classification problems in comparison with core Orange.

However — and this holds for core Orange as well as for text addon — if you would like to perform classification one feature must be set as class variable. Otherwise Orange cannot know which feature you would like to predict. When you are reading the data from CSV by default no feature is selected as class variable and hence the message.

There are two possible approaches how one can mark a feature as class variable. The more permanent one, as you already figured, is to modify the data file itself. Second, a bit more user friendly one, is to do this in Orange. To do so you just pass the data to Select Columns widget, drag the desired feature to the Target Variable field and click Send. The data on the output of the widget will have a class variable and can be passed to Test & Score. The following scheme should work without modifying the header of a file:

If you would like to know more about how the Select Columns widget works you can also check its documentation.

So in a nutshell you were missing a step and what happened is expected. If the header does not state that some feature is a class variable — as was the case for your CSV — Orange simply won't mark any feature as class variable. We don't use any heuristics to guess what might be a class variable but solely rely on what is written in the file.