0
$\begingroup$

I have been trying to build a fake news classifier DL model. Notebook here

Problem is when i put the same data in a Multinomial model it gives good accuracy. But when coverted into tensors and fed to DL model accuracy lingers around 50% . I tried tweaking some parameters and watched the dataset again and again. I couldn't figure out. I may me missing something somewhere.

I would love to have you views on where I am making the mistake. Notebook here

model1 = keras.Sequential()
model1.add(keras.Input(shape = (1), dtype = tf.string))
model1.add(title_text_vectorizer)
model1.add(title_embedder)
model1.add(layers.Conv1D(32, 5, 1, activation='relu'))
model1.add(layers.Dropout(.3))
model1.add(layers.Dense(32, activation='relu'))
model1.add(layers.Dense(1))

#compile
model1.compile(loss = tf.keras.losses.BinaryCrossentropy(),
              optimizer = tf.keras.optimizers.Adam(),
              metrics = ['accuracy'])

fitting

hist1 = model1.fit(train_title_dataset, epochs = EPOCHS,
                      steps_per_epoch = int(.3* (len(train_title_dataset)/EPOCHS)),
                      validation_steps = int(.2* (len(train_title_dataset)/EPOCHS)),
                      validation_data=test_title_dataset,
                      callbacks = [create_tensorboard_callback('tb','model1')]
                    )

output

Saving TensorBoard log files to: tb/model1/20221110-212802
Epoch 1/10
17/17 [==============================] - 0s 6ms/step - loss: 0.7065 - accuracy: 0.5051 - val_loss: 0.6983 - val_accuracy: 0.4963
Epoch 2/10
17/17 [==============================] - 0s 7ms/step - loss: 0.7057 - accuracy: 0.5052 - val_loss: 0.6992 - val_accuracy: 0.4964
Epoch 3/10
17/17 [==============================] - 0s 6ms/step - loss: 0.7062 - accuracy: 0.4980 - val_loss: 0.6973 - val_accuracy: 0.4970
Epoch 4/10
17/17 [==============================] - 0s 6ms/step - loss: 0.7062 - accuracy: 0.4988 - val_loss: 0.6975 - val_accuracy: 0.4967
Epoch 5/10
17/17 [==============================] - 0s 6ms/step - loss: 0.7042 - accuracy: 0.5010 - val_loss: 0.6965 - val_accuracy: 0.4998
Epoch 6/10
17/17 [==============================] - 0s 6ms/step - loss: 0.7027 - accuracy: 0.5060 - val_loss: 0.7022 - val_accuracy: 0.4966
Epoch 7/10
17/17 [==============================] - 0s 6ms/step - loss: 0.7009 - accuracy: 0.5165 - val_loss: 0.6993 - val_accuracy: 0.4961
Epoch 8/10
17/17 [==============================] - 0s 6ms/step - loss: 0.6997 - accuracy: 0.5158 - val_loss: 0.6971 - val_accuracy: 0.4965
Epoch 9/10
17/17 [==============================] - 0s 6ms/step - loss: 0.7032 - accuracy: 0.5011 - val_loss: 0.6959 - val_accuracy: 0.4997
Epoch 10/10
17/17 [==============================] - 0s 7ms/step - loss: 0.7043 - accuracy: 0.4999 - val_loss: 0.6973 - val_accuracy: 0.4966
$\endgroup$
4
  • $\begingroup$ Welcome to DataScienceSE. If a simple model works well and a complex one doesn't, my first guess would be overfitting. $\endgroup$
    – Erwan
    Nov 10, 2022 at 15:55
  • $\begingroup$ I am quite feeling welcomed. But please have a look if you can. Yell at my mistake. Please. $\endgroup$
    – tikendraw
    Nov 10, 2022 at 16:51
  • $\begingroup$ Had a very quick look, the first thing I would investigate is the NaN loss values during training. Apparently something goes wrong in the calculation of the loss, which could prevents the model from learning anything. $\endgroup$
    – Erwan
    Nov 10, 2022 at 18:19
  • $\begingroup$ Thank you @Erwan. You have been helpful. $\endgroup$
    – tikendraw
    Nov 11, 2022 at 1:01

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.