All classification models except neural network giving 100% accuracy

Question

I have a dataset of size (140,000, 10) containing 1 dependent variable. I used MinMax scaler on independent variables. For the target value, there is a class imbalance of 94% 0's and 6% 1's. Used RandomOverSampler for oversampling.

When I run a neural network model on my dataset, it gives me the following plot:

With a testing accuracy of 96.34%

This is my neural network:

model = models.Sequential()
model.add(layers.Dense(5, kernel_regularizer=regularizers.l2(0.001),
                       activation='relu', input_shape=(10,)))
model.add(layers.Dense(5, kernel_regularizer=regularizers.l1(0.001),
                       activation='relu'))
model.add(layers.Dense(10, kernel_regularizer=regularizers.l2(0.001), 
                       activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='adam', metrics = ['accuracy'])

history = model.fit(X_over, y_over, epochs=25, batch_size=100, validation_split=0.2, shuffle=True, verbose=1)

I have added regularizers to input and hidden layers to prevent overfitting.

On using a Random Forest Classifier and SVM, I get 100% accuracies for training as well as testing. Is something wrong with my data? How do I solve this?

Answer 1

It could be due to a lack of initialization of your neurons.

Did you initialize them randomly?

For instance:

from tensorflow.keras import layers
from tensorflow.keras import initializers
    
model.add(layers.Dense(5, kernel_regularizer=regularizers.l1(0.001), kernel_initializer=initializers.RandomNormal(stddev=0.01),activation='relu'))

Without a good initialization, your neurons fails to differentiate between each other properly during training, and cannot learn.

On the other hand, your validation dataset could be too small to be representative enough of your training dataset for NN. It could work better with a larger validation dataset or a different random distribution (using random seed).