0
$\begingroup$

My friend gave me this puzzle awhile ago and I've never figured it out.

import numpy as np
from sklearn.model selection import train test split
from sklearn.linear model import LogisticRegression
from sklearn.metrics import accuracy_score

X, y = get_data( )
X_train, X_test, y_train, y_test = train_test_split(x, y)

print(f"X_train shape: {X train.shape}")
# prints X train shape: (3000, 4)

print(f"X test shape: {X test.shape}")
# prints X test shape: (1000, 4)

print(f"y_train shape: {y_train.shape}")
# prints y_train shape: (3000, 1)

print(f"y test shape: {yـtest.shape}")
# prints y test shape: (1000, 1)

model = LogisticRegression()
model.fit(X_train, y_train)
y_predict = model.predict(Xـtest)

accuracy = accuracy_score(y_true=y_test, y_pred=y_predict)
print(accuracy)
## prints 0.947

print(pd.Series(y_test.reshape(-1)). value counts())
# prints:
#1  949
#0   51
# dtype: int64

The premise is the following: the person was working on a binary classification task. The model’s accuracy during development was high but it didn't perform well in real life. So they think the issue is with the training process. What is it?

I've been looking at it for a while now but I have no idea what to say. The imports are correct, calling the model is correct, fitting and accuracy measurements are correct. The only possible issues I see is that maybe the split is too big? Any thoughts/hints on this?

Improve this question
$\endgroup$
6
  • $\begingroup$ There is no X test, there is X_test $\endgroup$
    – Nikos M.
    Jul 11, 2022 at 18:08
  • $\begingroup$ Same for y test there is y_test $\endgroup$
    – Nikos M.
    Jul 11, 2022 at 18:08
  • $\begingroup$ Except the above typos, what remains is a class imbalance which would require special handling $\endgroup$
    – Nikos M.
    Jul 11, 2022 at 18:15
  • $\begingroup$ I fixed the typos. Thanks! Class imbalance in the labels right? How do you handle that? $\endgroup$
    – Muddy Penguin
    Jul 11, 2022 at 18:24
  • 1
    $\begingroup$ I think using accuracy, especially without any view into the optimal cut off, is an error. Why use the default cut-off? Some more info - stats.stackexchange.com/questions/312780/…. And take a look at the first comment here for more info that many times class imbalance is not a problem, especially when using logistic regression. datascience.stackexchange.com/questions/90964/…. Imbalance may or may not be an issue 0 you need to research your model. $\endgroup$
    – Craig
    Jul 12, 2022 at 9:38

1 Answer 1

Reset to default
-1
$\begingroup$

You should Include Test Size .For Example :To set Test Size 20% and also fix the typos .

X_train,X_test,y_train,y_test=train_test_split(x, y,test_size=0.2)
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.