# A Loss of 55.2164 with a sparse_categorical_crossentropy in a sequential neural network?

I'm following Aurélion Géron's book on Machine Learning. The following code tries to evaluate a neural network with a sparse categorical cross entropy loss function, on the Fashion Mnist data set. How come I get such a strange value for loss?

import tensorflow as tf
from tensorflow import keras
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

fashion_mnist=keras.datasets.fashion_mnist

X_valid , X_train = X_train_full[:5000]/255.0 , X_train_full[5000:]/255.0
y_valid , y_train = y_train_full[:5000] , y_train_full[5000:]

model=keras.models.Sequential()

model.compile(loss="sparse_categorical_crossentropy",optimizer="sgd",metrics=["accuracy"])
history=model.fit(X_train,y_train,epochs=30,validation_data=(X_valid,y_valid))

#some instructions with *history* for plotting a graph

model.evaluate(X_test,y_test)


and the output I get is [55.21640347443819, 0.8577]

How come I get a loss over 1?

• For those easy trigger users, could you just give me a suggestion for improvement, instead of just downvoting the question? Thanks. Mar 28 '20 at 9:54
• I added an answer to explain the root cause of this issue in datascience.stackexchange.com/questions/38399/… Jul 25 '20 at 18:15

The reason you are getting a very high loss has to do with the normalization you are performing. You are normalizing both the X_train and X_valid, however you do not normalize X_valid. Adding normalization of X_valid to your code the final loss for X_valid is much lower.

fashion_mnist = keras.datasets.fashion_mnist

(X_train_full,y_train_full), (X_test,y_test) = fashion_mnist.load_data()

X_valid , X_train = X_train_full[:5000]/255.0 , X_train_full[5000:]/255.0
y_valid , y_train = y_train_full[:5000] , y_train_full[5000:]

# Normalizing X_test as well
X_test = X_test/255.0

model = keras.models.Sequential()