I'm working with this dataset https://www.kaggle.com/c/sf-crime to predict the crime incident using keras. I've encoded the category with pd.get_dummies and then use it as the validation data. At first I try to use categorical_crossentropy and adam for the loss function but result returns with very bad acc which nearly 0.3 and about 2.7 of loss. So I try to experiment with different entropy by using binary_crossentropy. The result seem to be exaggerate the acc is higher than 0.97 and the loss is less than 0.1 at first epoch. I also provide 0.3 of drop out for each layer. So which crossentropy is accurate or better use for this scenario and which kind of this classification is multi-label or multi-class?
1 Answer
$\begingroup$
$\endgroup$
which kind of this classification is multi-label or multi-class?
If we take a look at the data panel at the competition's page in kaggle:
Category - category of the crime incident (only in train.csv). This is the target variable you are going to predict.
The target we are trying to predict is the category of the crime (which can take multiple values) so this answers your question: the problem is multi-class.
So which crossentropy is accurate or better use for this scenario?
There are three main types of cross-entopy in keras:
- categorical cross-entropy is used for multi-class classification, where the target variable is one-hot encoded.
- sparse categorical cross-entropy is used for multi-class classification, where the target variable is not one-hot encoded.
- binary cross-entropy is used for binary classification problems.
From your question I assume that you think the different types of crossentropy are interchangeable. That is not the case. You should select the cross-entropy based on how your problem and how you encoded your data.
In your case, since you have a multi-class problem and you state that you have used pandas.get_dummies(), which ont-hot encodes your data you should use categorical cross-entropy and nothing else!