2
$\begingroup$

I am currently collecting second by second data regarding buyer vs seller initiated trades for different financial instruments (securities mostly). If there are more buyer initiated trades in a given second, then that second's data point would contain a positive value in the pertinent feature. If there are more seller initiated trades, then there would be a negative value. And if either there is an equal amount of buy vs seller initiated trades OR if there are simply not any trades in a given second, there will be a 0 for the feature in that data point. Along with this feature, there are several other features that are based on what occurred in the preceding seconds (eg if the value discussed above was 12 for the data point immediately preceding the current point, then the second feature for the current data point would be 12 - please let me know if this is not clear) After much troubleshooting, I have concluded that if there are too many data points with too many 0's for features, the classifier simply wont work. When I print out the probabilities of evaluation data points falling into different classes, I simply get

0:NaN,1:NaN

for all model evaluation points I try to classify. (I am using logistic regression from apache-mahout. In total have 183 features, but over 40million data points. There are three categories to which the data point can be classified)

I have found that if I set the default value to 1, then I no longer encounter this error, e.g. if there are no trades, the value will be a 1, if there is one seller initiated trade, the value will be 0.

So with all this in mind, I have two related questions:

1) Has anyone else encountered this issue? e.g. if you have a vector with x features, and for a majority of the data points, a majority of the features contain 0's, is this know to give issues?

2) Is shifting all values up by a constant (such as 1) a valid fix to this issue? I assume that if this constant is applied to all values, then it shouldn't skew the data, but I figure it won't hurt to check with the experts.

Also, I'm new to this, so if you believe that my question could use more info please let me know, and if you could give me ideas of what information to include, it would be greatly appreciated.

thanks in advance

Improve this question
$\endgroup$
5
  • $\begingroup$ What classifier are you using? $\endgroup$
    – shadowtalker
    Commented Jan 11, 2015 at 15:43
  • $\begingroup$ logistic regression in apache mahout $\endgroup$
    – user3256725
    Commented Jan 11, 2015 at 17:28
  • $\begingroup$ The problem is not with zeroes per se but with too many of one class and not enough of the other; logistic regression is usually fitted with maximum likelihood (although I can't speak specifically for the Mahout implementation) and that causes problems for the algorithm. A search for "rare events logistic regression" on the Stats.StackExchange might help you. I'd write a bigger answer but right now I'm posting from my phone and only have a few minutes $\endgroup$
    – shadowtalker
    Commented Jan 11, 2015 at 18:31
  • $\begingroup$ Are the vectors where you encounter this error all zeroes? If so, and you're not fitting an intercept/bias term, this is your problem $\endgroup$
    – Ben Allison
    Commented Mar 13, 2015 at 13:30
  • $\begingroup$ And you could fix it by simply including a bias $\endgroup$
    – Ben Allison
    Commented Mar 13, 2015 at 13:30

1 Answer 1

Reset to default
1
$\begingroup$

From your question I am not sure how you are using logistic regression. The "vanilla" version of logistic regression gives a binary classifier, but your target has three values: a positive value, a negative value, and zero. The Apache site, on the page for logistic regression refers to the thesis of Paul Komarek, in which "logistic regression" refers to the binary classifier. So your problem might be that there your target variable has 3 values.

Assuming that that is the case, you have many options, for example: - a) use "multi-nomial logistic", - b) use ordinary logistic regression 3 times to fit 3 models, one per level, to distinguish it from the other two combined

Improve this answer
$\endgroup$
1
  • $\begingroup$ The apache mahout does indeed support three class classifiers. And the positive negative values were features, but I will indeed give multiple classifiers a shot. Thanks for the input $\endgroup$
    – user3256725
    Commented Jan 12, 2015 at 11:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.