0
$\begingroup$

I am new to neural networks and have done a few projects but have got very low accuracy for all of them. I have included the code for titanic NN code here. Am I missing something or what? Can you help me with this?

'''

import numpy as np
import pandas as pd

train=pd.read_csv(r'E:\Learning Python\Kaggle Competition\TItanic\Dataset\train.csv')
test=pd.read_csv(r'E:\Learning Python\Kaggle Competition\TItanic\Dataset\test.csv')

train.head()

train.iloc[[60]]

train.info()

train.describe()

train.drop(['Name','Ticket','Cabin', 'PassengerId'], axis=1, inplace=True)

categorical_cols=[]
numeric_cols=[]

for col in train.columns:
    if train[col].dtype=='object':
        categorical_cols.append(col)
    else:
        numeric_cols.append(col)

print(categorical_cols)


print(numeric_cols)

for col in categorical_cols:
    print(train[col].unique())

train['Survived'].unique()

train.isna().sum()

train.describe()

### Handling Missing Values

train['Age']=train['Age'].fillna(train['Age'].median())

train=train[~train['Embarked'].isna()]
train=train[~train['Survived'].isna()]

for col in categorical_cols:
    train[col]=train[col].astype('category')

train.dtypes

### Encoding Categorical Values

categorical_cols

for col in categorical_cols:
    print(train[col].unique())
    print(train[col].isna().sum())



from sklearn.preprocessing import OneHotEncoder

ohe=OneHotEncoder(drop='first')

#for Sex column
encoded_array=ohe.fit_transform(train['Sex'].values.reshape(-1,1)).toarray()

encoded_df=pd.DataFrame(encoded_array, columns=ohe.get_feature_names_out(['Sex']))
encoded_df.shape


train.shape

train = train.reset_index(drop=True)
encoded_df = encoded_df.reset_index(drop=True)
train=pd.concat([train, encoded_df], axis=1)
train=train.drop(['Sex'], axis=1)

train.isna().sum()

# for Embarked column
encoded_array=ohe.fit_transform(train['Embarked'].values.reshape(-1,1)).toarray()
encoded_df=pd.DataFrame(encoded_array, columns=ohe.get_feature_names_out(['Embarked']))
train=pd.concat([train, encoded_df], axis=1)
train=train.drop(['Embarked'], axis=1)

train.head()

### Splitting train and test data

from sklearn.model_selection import train_test_split

X=train.drop(['Survived'], axis=1)
y=train['Survived']

X_train, X_test, y_train, y_test=train_test_split(X,y,test_size=0.2, random_state=1, shuffle=True)

train.isna().sum()





### Scaling Numeric Values

from sklearn.preprocessing import StandardScaler, PowerTransformer

pt=PowerTransformer()

categorical_cols=[]
numeric_cols=[]

for col in X_train.columns:
    if train[col].dtype=='object':
        categorical_cols.append(col)
    else:
        numeric_cols.append(col)

X_train[numeric_cols]=pt.fit_transform(X_train[numeric_cols])

X_test[numeric_cols]=pt.transform(X_test[numeric_cols])

#st=StandardScaler()
#X_train[numeric_cols]=st.fit_transform(X_train[numeric_cols])
#X_test[numeric_cols]=st.transform(X_test[numeric_cols])

# Data Visualization

import seaborn as sns
import matplotlib.pyplot as plt

plt.figure(figsize=(15,15))
sns.heatmap(train.corr(), cmap='jet', annot=True, linewidth=True)
plt.show()

plt.figure(figsize=(20,10))
sns.boxplot(train[numeric_cols])
plt.xticks(rotation=30)
plt.show()

## Neural Network

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Sequential, optimizers
from tensorflow.keras import layers
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.layers import Dense, Dropout, BatchNormalization
from tensorflow.keras.optimizers import Adam, SGD
from sklearn.metrics import accuracy_score

model=Sequential()
model.add(Dense(256, activation='relu', input_dim=X_train.shape[1]))
model.add(Dense(32, activation='relu'))
model.add(Dense(4, activation='relu'))
model.add(Dense(1, activation='softmax'))

model.compile(optimizer='SGD', loss='binary_crossentropy', metrics='Accuracy')

model.fit(X_train, y_train, epochs=100, validation_split=0.2)

'''

$\endgroup$
1
  • $\begingroup$ Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. $\endgroup$
    – Community Bot
    Jul 15, 2023 at 9:15

1 Answer 1

0
$\begingroup$

Your model will always give output of Class 0 since you are using softmax activation function and 1 output node for binary classification. The output of a Softmax is a vector with probabilities of each possible outcome. So, the softmax layer should have the same number of nodes as the number of classes. You need to change the number of nodes in output layer to 2 and also change y_train accordingly (one-hot encoding)

OR

You can keep the number of nodes in output layer as 1, and change the activation function of output layer to sigmoid. If the output of your sigmoid output node will be low, it will assign "Class 0", else "Class 1". Softmax is basically an extension of sigmoid.
You can go through this article for better understanding of the two functions.

Note: For binary classification, one output node with sigmoid activation function is preferred as it will update faster due to less number of parameters.

$\endgroup$
3
  • $\begingroup$ Thank you got it. I had another problem where I've used three categories for the output, converted the y values in the array of (X,3) shape with binary values in each row. I've used the softmax function but I see that the prediction always predicts the second category giving around 33% accuracy. The probabilities of all three categories are in the range of 0.3 with the second category always having the highest. Is there any particular reason for that? $\endgroup$ Jul 18, 2023 at 3:50
  • $\begingroup$ Please show your code, if possible. $\endgroup$
    – shivani
    Jul 18, 2023 at 5:39
  • $\begingroup$ transfernow.net/dl/20230719hOts20h0 The code and the data csv files are here. Thank you. $\endgroup$ Jul 19, 2023 at 8:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.