activation=tf.keras.activations.relu vs activation='relu'

Question

Both models are for binary classification problems

Model 1

tf.random.set_seed(42)

model_1 = tf.keras.Sequential([
    tf.keras.layers.Dense(4, activation='relu'),
    tf.keras.layers.Dense(4, activation='relu'),
    tf.keras.layers.Dense(1)
])


model_1.compile(loss='binary_crossentropy',
                optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
                metrics=['accuracy'])


history = model_1.fit(X, y, epochs=250)

Model 2

tf.random.set_seed(42)

model_2 = tf.keras.Sequential([
    tf.keras.layers.Dense(4, activation=tf.keras.activations.relu),
    tf.keras.layers.Dense(4, activation=tf.keras.activations.relu),
    tf.keras.layers.Dense(1)
])


model_2.compile(loss='binary_crossentropy',
                optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
                metrics=['accuracy'])


history = model_2.fit(X, y, epochs=250)

Both models perform differently, model 1 performs with 0.5 accuracy and model 2 performs with 0.85-0.9ish accuracy. I'm confused why it's different. For model 1, I'd say the recent why it's performing bad is because I didn't define the output activation function. However, why does model 2 perform well despite not including the output activation function?

Answer 1

Your results were probably result of the randomness in the training. Both activation=tf.keras.activations.relu and activation='relu' are equivalent.

To obtain reproducible results, consider using tf.keras.utils.set_random_seed instead of tf.random.set_seed, because the latter is not enough (see this).