1
$\begingroup$

Is there a special header structure required for text classification problems?

I'm trying to do Tweet classification using the text mining plugin. I have the data in a CSV file, with one column labeled category and containing my two categories, and the other labeled text containing the tweets.

When I load the file with the corpus widget and follow the workflow given at http://orange3-text.readthedocs.io/en/latest/widgets/bagofwords.html the test widget gives the error "Train data input requires a class variable."

I opened the sample bookexcerpts.tab data and discovered it has a different header structure than just category and text. If I modify my tweet CSV to match it seems to work. Is this a bug, or am I missing a step?

$\endgroup$
3
$\begingroup$

No, no special structure is required for text classification problems in comparison with core Orange.

However — and this holds for core Orange as well as for text addon — if you would like to perform classification one feature must be set as class variable. Otherwise Orange cannot know which feature you would like to predict. When you are reading the data from CSV by default no feature is selected as class variable and hence the message.

There are two possible approaches how one can mark a feature as class variable. The more permanent one, as you already figured, is to modify the data file itself. Second, a bit more user friendly one, is to do this in Orange. To do so you just pass the data to Select Columns widget, drag the desired feature to the Target Variable field and click Send. The data on the output of the widget will have a class variable and can be passed to Test & Score. The following scheme should work without modifying the header of a file:

enter image description here

If you would like to know more about how the Select Columns widget works you can also check its documentation.

So in a nutshell you were missing a step and what happened is expected. If the header does not state that some feature is a class variable — as was the case for your CSV — Orange simply won't mark any feature as class variable. We don't use any heuristics to guess what might be a class variable but solely rely on what is written in the file.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for helping me with the selection widget--that makes sense. Unfortunately, when I try it I get a new error: TypeError: ufunc 'isnan' not supported for the input types. No matter, my workaround is functioning, and now I know how to use the Select Columns widget tool too. $\endgroup$ – Bryan Lee Feb 7 '17 at 23:34
  • $\begingroup$ Great! Regarding the TypeError, it seems like a bug. If you can provide us with a stack trace and a schema I can take a look and fix it. Maybe you can just open an issue at our issue tracker and we can continue the discussion there? $\endgroup$ – nikicc Feb 8 '17 at 14:26
  • $\begingroup$ Okay, I'll do that. Thanks again for the response and walk through--really helped. $\endgroup$ – Bryan Lee Feb 9 '17 at 0:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.