Accuracy of the model

Question

I'm using this dataset and i'm trying to do logistic regression

heart_data = pd.read_csv('../input/heart-disease-uci/heart.csv')
X = heart_data.iloc[:,:-1]
y = heart_data.iloc[:,-1]
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.20,random_state=5,shuffle=True)

from sklearn.preprocessing import MinMaxScaler

nr = MinMaxScaler()
X_train = nr.fit_transform(X_train)
X_test = nr.transform(X_test)

from sklearn.metrics import accuracy_score

def cal(method,c=1):
    lr = LogisticRegression(C=c,solver=method)
    lr.fit(X_train,y_train)
    pre_test = lr.predict(X_test)
    pre_train = lr.predict(X_train)
    train_score = accuracy_score(y_train,pre_train)
    test_score = accuracy_score(y_test,pre_test)
    return train_score,test_score
for i in method:
    print(i,'->>>>>>>>',cal(i))

First is training accuracy and second one is testing accuracy. why am i getting more testing accuracy over training?

And Is there another way to increase the both accuracy? I'm using min-max scaling so are there any other normalization to increase the accuracy more or this is the best accuracy we can get using logistic?

Answer 1

Good that you raise the question because there's most likely a bug.

I've done over 100 ML models and never seen test accuracy being higher than train accuracy.

Potential bug? Most likely there are much more class labels in your training than in your test set. Solution? Try in class_weight = "balanced". This most likely fixes it.

Another potential problems? Did you fix the random seed? You did not set C at a high level? E.g. 100, 1000..

PS: Nice! I see you normalize first and then split train and test! Well done. By doing so, you avoid data leakage.

Answer 2

Having a higher accuracy on the test set, than the train set, isn't inherently bad. It means your model definitely did not overfit the training data and it generalized enough for unseen data. You can check many different metrics to verify. Accuracy is really only one piece of the puzzle.

As for increasing the accuracy, it will take some work. There's no guarantee but there are steps that can be taken in search of higher accuracy. Try some things like feature reduction where you use only the most important features. There are many techniques to do this. You can also try to tune your hyperparameters which is a bit tedious. The sklearn docs will show you all of the default values used for the model and you'll be able to tune accordingly. You can perform basic data analytics on the full dataset to help guide your decisions.