2
$\begingroup$

I have a dataset of size (140,000, 10) containing 1 dependent variable. I used MinMax scaler on independent variables. For the target value, there is a class imbalance of 94% 0's and 6% 1's. Used RandomOverSampler for oversampling.

When I run a neural network model on my dataset, it gives me the following plot: Accuracy and Loss plot

With a testing accuracy of 96.34%

This is my neural network:

model = models.Sequential()
model.add(layers.Dense(5, kernel_regularizer=regularizers.l2(0.001),
                       activation='relu', input_shape=(10,)))
model.add(layers.Dense(5, kernel_regularizer=regularizers.l1(0.001),
                       activation='relu'))
model.add(layers.Dense(10, kernel_regularizer=regularizers.l2(0.001), 
                       activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='adam', metrics = ['accuracy'])

history = model.fit(X_over, y_over, epochs=25, batch_size=100, validation_split=0.2, shuffle=True, verbose=1)

I have added regularizers to input and hidden layers to prevent overfitting.

On using a Random Forest Classifier and SVM, I get 100% accuracies for training as well as testing. Is something wrong with my data? How do I solve this?

$\endgroup$

1 Answer 1

1
$\begingroup$

It could be due to a lack of initialization of your neurons.

Did you initialize them randomly?

For instance:

from tensorflow.keras import layers
from tensorflow.keras import initializers
    
model.add(layers.Dense(5, kernel_regularizer=regularizers.l1(0.001), kernel_initializer=initializers.RandomNormal(stddev=0.01),activation='relu'))

Without a good initialization, your neurons fails to differentiate between each other properly during training, and cannot learn.

On the other hand, your validation dataset could be too small to be representative enough of your training dataset for NN. It could work better with a larger validation dataset or a different random distribution (using random seed).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.