2
$\begingroup$

I'm trying to run a Random Forest model from sklearn but I keep getting an error: ValueError: Input contains NaN, infinity or a value too large for dtype('float32').

I tried following steps in ValueError: Input contains NaN, infinity or a value too large for dtype('float32')

fillna(0) on my pandas dataframe still gave the ValueError.

So I tried working with my numpy array:

val = setTo.ravel().nan_to_num(0)

But I keep getting an error: 'numpy.ndarray' object has no attribute 'nan_to_num'

I'm wondering how I can deal with the nan values if I have ndarray?

Update

Thanks so much to @Beniamin H for all the help, as suggested, I rescaled the data, which I based on https://stackoverflow.com/questions/34771118/sklearn-random-forest-error-on-input and it worked!

$\endgroup$
2
  • 1
    $\begingroup$ Hi and welcome to Data Science Stack Exchange :) $\endgroup$
    – Beniamin H
    Jan 6, 2021 at 20:17
  • 1
    $\begingroup$ Thanks so much:) $\endgroup$
    – YJay
    Jan 6, 2021 at 22:03

1 Answer 1

1
$\begingroup$

You are using the right method but in a wrong way :)

nan_to_num is a method of numpy module, not numpy.ndarray. So instead of calling nan_to_num on you data, call it on numpy module giving your data as a paramter:

import numpy as np
data = np.array([1,2,3,np.nan,np.nan,5])
data_without_nan = np.nan_to_num(data)

prints:

array([1., 2., 3., 0., 0., 5.])

In your example:

import numpy as np
val = np.nan_to_num(setTo.ravel())
$\endgroup$
12
  • $\begingroup$ Thanks so much! I tried val = np.nan_to_num(setTo.ravel()) but I still seem to be getting: ValueError: Input contains NaN, infinity or a value too large for dtype('float32'). $\endgroup$
    – YJay
    Jan 6, 2021 at 22:22
  • $\begingroup$ I'm also not sure if it's due to a NaN value or possibly a value too large. I don't seem to see any NaN in the actual data $\endgroup$
    – YJay
    Jan 6, 2021 at 22:26
  • $\begingroup$ You may try to use pandas replace: df.replace([np.inf, -np.inf], 0) to replace inf and -inf with 0 $\endgroup$
    – Beniamin H
    Jan 6, 2021 at 22:58
  • $\begingroup$ There are more answers which may help you: stackoverflow.com/questions/31323499/… $\endgroup$
    – Beniamin H
    Jan 6, 2021 at 23:00
  • $\begingroup$ Thanks again for your help! So I tried df.replace([np.inf, -np.inf], 0) but still getting the error. I tried some of the solutions in stackoverflow.com/questions/31323499/…, np.isfinite(X) showed true so that must be the problem!! $\endgroup$
    – YJay
    Jan 6, 2021 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.