0
$\begingroup$
def preprocess_features(X):
    output = pd.DataFrame(index = X.index)
    for col , col_data in X.iteritems():
        if col_data.dtype == object :
            col_data = pd.get_dummies(col_data , prefix = col)
        output = output.join(col_data)
    return output
X_a = preprocess_features(X_all)


X_train , X_test , Y_train , Y_test = train_test_split(X_a , Y_all , test_size = 0.15, random_state = 2)
#print(X_train.shape,X_test.shape,Y_train.shape,Y_test.shape)
clf = LogisticRegression()
clf.fit(X_train , Y_train)

When I run this code, I get the following error: How do I resolve it?

ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

$\endgroup$
  • $\begingroup$ First, check for column with values zero in it. If you have implemented any log() transformation on such column, it would have lead to this error while training. $\endgroup$ – DataFramed Mar 4 at 13:48
1
$\begingroup$

Check the columns inside your dataframe.

You can try:

X.count()

to see how many data entries each column has.

From there, you can try:

.fillna()

or go back to your data and fill the necessary information or use sklearn.preprocessing.Imputer encode mean / median imputation for missing values.

Make sure that each column doesn't have any string or letter in it. Check also each column and get their max value, maybe you can convert them in much more smaller number (e.g. Normalizing)

Also, try this:

np.isnan(X)  

to get a boolean form of your dataframe to check if there's any blanks

| improve this answer | |
$\endgroup$
  • $\begingroup$ I was having a similar problem so I implemented the .fillna(0, inplace = True) which still returned the same error. I tried finding the NaN values by running: np.isnan(df).sum() which returned all columns as 0. I inferred that there are no NaN values but it still gives the same error. $\endgroup$ – Ammanuel May 31 at 11:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.