1
$\begingroup$

Since my data is too large, I use pd.read_csv('',chunksize=). I am using categorical_crossentropy as my loss function, however, on the last chunk, I have just one target. So I get the error:

You are passing a target array of shape (2110, 1) while using as loss categorical_crossentropy.

Now I know I can use binary_crossentropy.So this is what I did:

X_train, X_test, y_train, y_test = train_test_split(train_data, train_labels, shuffle=True, test_size=0.3)
if y_train.shape[1] == 1:
   loss = 'binary_crossentropy'
else:
   loss = 'categorical_crossentropy'

When I do this, I get the error:

IndexError: index 1 is out of bounds for axis 0 with size 1

My data is one-hot encoded. How can I resolve this error? Thanks

Improve this question
$\endgroup$
4
  • $\begingroup$ Shuffle your data simple! $\endgroup$
    – Aditya
    Commented Mar 26, 2020 at 12:11
  • $\begingroup$ Thank you for your response. I do shuffle my data in the train test split. Is that what you're talking about? $\endgroup$
    – Anan Srivastava
    Commented Mar 26, 2020 at 12:21
  • $\begingroup$ Any suggestion? $\endgroup$
    – Anan Srivastava
    Commented Mar 26, 2020 at 20:08
  • $\begingroup$ Well we will need a minimal reproducible example to help you further! But I feel error means something else :) (try checking it out!) $\endgroup$
    – Aditya
    Commented Mar 27, 2020 at 2:03

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.