Very low Neural Network Accuracy for Titanic Survival Problem

Question

I am new to neural networks and have done a few projects but have got very low accuracy for all of them. I have included the code for titanic NN code here. Am I missing something or what? Can you help me with this?

'''

import numpy as np
import pandas as pd

train=pd.read_csv(r'E:\Learning Python\Kaggle Competition\TItanic\Dataset\train.csv')
test=pd.read_csv(r'E:\Learning Python\Kaggle Competition\TItanic\Dataset\test.csv')

train.head()

train.iloc[[60]]

train.info()

train.describe()

train.drop(['Name','Ticket','Cabin', 'PassengerId'], axis=1, inplace=True)

categorical_cols=[]
numeric_cols=[]

for col in train.columns:
    if train[col].dtype=='object':
        categorical_cols.append(col)
    else:
        numeric_cols.append(col)

print(categorical_cols)


print(numeric_cols)

for col in categorical_cols:
    print(train[col].unique())

train['Survived'].unique()

train.isna().sum()

train.describe()

### Handling Missing Values

train['Age']=train['Age'].fillna(train['Age'].median())

train=train[~train['Embarked'].isna()]
train=train[~train['Survived'].isna()]

for col in categorical_cols:
    train[col]=train[col].astype('category')

train.dtypes

### Encoding Categorical Values

categorical_cols

for col in categorical_cols:
    print(train[col].unique())
    print(train[col].isna().sum())



from sklearn.preprocessing import OneHotEncoder

ohe=OneHotEncoder(drop='first')

#for Sex column
encoded_array=ohe.fit_transform(train['Sex'].values.reshape(-1,1)).toarray()

encoded_df=pd.DataFrame(encoded_array, columns=ohe.get_feature_names_out(['Sex']))
encoded_df.shape


train.shape

train = train.reset_index(drop=True)
encoded_df = encoded_df.reset_index(drop=True)
train=pd.concat([train, encoded_df], axis=1)
train=train.drop(['Sex'], axis=1)

train.isna().sum()

# for Embarked column
encoded_array=ohe.fit_transform(train['Embarked'].values.reshape(-1,1)).toarray()
encoded_df=pd.DataFrame(encoded_array, columns=ohe.get_feature_names_out(['Embarked']))
train=pd.concat([train, encoded_df], axis=1)
train=train.drop(['Embarked'], axis=1)

train.head()

### Splitting train and test data

from sklearn.model_selection import train_test_split

X=train.drop(['Survived'], axis=1)
y=train['Survived']

X_train, X_test, y_train, y_test=train_test_split(X,y,test_size=0.2, random_state=1, shuffle=True)

train.isna().sum()





### Scaling Numeric Values

from sklearn.preprocessing import StandardScaler, PowerTransformer

pt=PowerTransformer()

categorical_cols=[]
numeric_cols=[]

for col in X_train.columns:
    if train[col].dtype=='object':
        categorical_cols.append(col)
    else:
        numeric_cols.append(col)

X_train[numeric_cols]=pt.fit_transform(X_train[numeric_cols])

X_test[numeric_cols]=pt.transform(X_test[numeric_cols])

#st=StandardScaler()
#X_train[numeric_cols]=st.fit_transform(X_train[numeric_cols])
#X_test[numeric_cols]=st.transform(X_test[numeric_cols])

# Data Visualization

import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(15,15))
sns.heatmap(train.corr(), cmap='jet', annot=True, linewidth=True)
plt.show()

plt.figure(figsize=(20,10))
sns.boxplot(train[numeric_cols])
plt.xticks(rotation=30)
plt.show()

## Neural Network

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Sequential, optimizers
from tensorflow.keras import layers
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.layers import Dense, Dropout, BatchNormalization
from tensorflow.keras.optimizers import Adam, SGD
from sklearn.metrics import accuracy_score

model=Sequential()
model.add(Dense(256, activation='relu', input_dim=X_train.shape[1]))
model.add(Dense(32, activation='relu'))
model.add(Dense(4, activation='relu'))
model.add(Dense(1, activation='softmax'))

model.compile(optimizer='SGD', loss='binary_crossentropy', metrics='Accuracy')

model.fit(X_train, y_train, epochs=100, validation_split=0.2)

'''

Answer 1

Your model will always give output of Class 0 since you are using softmax activation function and 1 output node for binary classification. The output of a Softmax is a vector with probabilities of each possible outcome. So, the softmax layer should have the same number of nodes as the number of classes. You need to change the number of nodes in output layer to 2 and also change y_train accordingly (one-hot encoding)

OR

You can keep the number of nodes in output layer as 1, and change the activation function of output layer to sigmoid. If the output of your sigmoid output node will be low, it will assign "Class 0", else "Class 1". Softmax is basically an extension of sigmoid.
You can go through this article for better understanding of the two functions.

Note: For binary classification, one output node with sigmoid activation function is preferred as it will update faster due to less number of parameters.