I'n struggling with categorical_crossentropy problem with one-hot encoding data. The problem is in unchanged output of code presenting below:
inputs = keras.Input(shape=(1190,), sparse=True)
lay_1 = layers.Dense(1190, activation='relu')
x = lay_1(inputs)
x = layers.Dense(10, activation='relu')(x)
out = layers.Dense(1, activation='sigmoid')(x)
self.model = keras.Model(inputs, out, name='SimpleD2Dense')
self.model.compile(
optimizer=keras.optimizers.Adam(),
loss=tf.losses.categorical_crossentropy,
metrics=['accuracy']
)
Epoch 1/3
1572/1572 - 6s - loss: 5.7709e-08 - accuracy: 0.5095 - val_loss: 7.0844e-08 - val_accuracy: 0.5543
Epoch 2/3
1572/1572 - 6s - loss: 5.7709e-08 - accuracy: 0.5095 - val_loss: 7.0844e-08 - val_accuracy: 0.5543
Epoch 3/3
1572/1572 - 7s - loss: 5.7709e-08 - accuracy: 0.5095 - val_loss: 7.0844e-08 - val_accuracy: 0.5543
Few words about data: 1190 features (10 actual features with 119 categories). The inputs are a dataframe rows with 1190 values per sample. Output is a binary value 0 or 1.
Attempts done before: binary_crossentropy used with satisfying results, however, number of samples is not enough to get good results on validation data. Tried to use different activations and layer sizes.
Main question is why categorical_crossentropy is not working and how to use it in right way.
Also, one concern appears about data representation is it right way to use in one rare row of straightforward one-hot encoded data?