I am trying to improve the accuracy of my model over the UCI Breast Cancer Dataset. There's 426 records, and it is a binary classification model.
X_train.shape
(426, 30)
X_test.shape
(143, 30)
y_train.shape
(426, 1)
y_test.shape
(143, 1)
- I have tried three types of gradient descents to compare the accuracies and it remains stagnant on 0.62.
- Batch sizes: Batch gradient descent over the whole dataset, for stochastic it is 1/1 and mini batch takes 16 batches. All of them show the same accuracy.
- The activation layers used are relu, relu, sigmoid, for 6,4,2 neurons in each layer.
- The loss function used is cross entropy.
- I have normalised the dataset using StandardScaler and Min Max Scaler from scikit learn.
- Learning rate is 0.00095, and epochs is 50. The following shows the learning rate after each epoch for stochastic gradient descent
Cost after epoch 0: 0.690206
Cost after epoch 10: 0.664892
Cost after epoch 20: 0.661360
Cost after epoch 30: 0.660859
Cost after epoch 40: 0.660782
Accuracy
0.6293706293706294
The Confusion Matrix gives 0 values for TN and FN, and I am aware this might be wrong, but do not understand where its going wrong.
90 0 53 0
Apart from all this, I have tried experimenting with different epoch numbers, hidden neurons, layers and learning rates, but accuracy does not improve.