1
$\begingroup$

I was trying to reduce the skewness of data using boxcox transformation. But was facing an error:

ValueError: Data must be positive.

I figured out why it was throwing an error, my data ranged from .0 to 1 so I added one and it worked.

Although is it the right way to deal with it?

$\endgroup$
1
  • 1
    $\begingroup$ Welcome to DS StackExchange. Please elaborate a more coherent question. It seems that you started with a problem, solved it, and converted it to another. Could you update your question to the new form? Right now, your title and your final questions are about two different things, generating inconsistency. Thank you $\endgroup$
    – Leevo
    Jan 8 '20 at 11:34
0
$\begingroup$

Boxcox you need to tune the lambda parameter, in order to transform the data to normal one (and than its implicitly non-skewed) however, I would suggest to rather use log transform, its faster.

$\endgroup$
5
  • $\begingroup$ Is it decent to use it for continuous data? My data ranges from .0 to 1 $\endgroup$
    – Swarley
    Jan 8 '20 at 12:18
  • $\begingroup$ Sure it is, try it $\endgroup$
    – Noah Weber
    Jan 8 '20 at 13:03
  • $\begingroup$ Well the problem with log transform is that the output is NaN since my data has 0's also. $\endgroup$
    – Swarley
    Jan 8 '20 at 13:10
  • $\begingroup$ check this out docs.scipy.org/doc/numpy/reference/generated/numpy.log1p.html $\endgroup$
    – Noah Weber
    Jan 8 '20 at 13:16
  • $\begingroup$ Thanks, tried, the skewness of the data is unaltered. The only thing that's working is adding one to the data and I am not sure whether it is a good approach or not. $\endgroup$
    – Swarley
    Jan 8 '20 at 13:21
0
$\begingroup$

Just look for the smallest non zero entry in your data, let this be e.g. x, then add x/2 to this smallest values and compute the boxcox. Adding a small value i.e epsilon, doesn't affect that much to our data, otherwise adding 1 to all value is also good strategy, you can check which one gives you better results. Here is a link for more info : https://discuss.analyticsvidhya.com/t/methods-to-deal-with-zero-values-while-performing-log-transformation-of-variable/2431/3

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.