My friend gave me this puzzle awhile ago and I've never figured it out.
import numpy as np
from sklearn.model selection import train test split
from sklearn.linear model import LogisticRegression
from sklearn.metrics import accuracy_score
X, y = get_data( )
X_train, X_test, y_train, y_test = train_test_split(x, y)
print(f"X_train shape: {X train.shape}")
# prints X train shape: (3000, 4)
print(f"X test shape: {X test.shape}")
# prints X test shape: (1000, 4)
print(f"y_train shape: {y_train.shape}")
# prints y_train shape: (3000, 1)
print(f"y test shape: {yـtest.shape}")
# prints y test shape: (1000, 1)
model = LogisticRegression()
model.fit(X_train, y_train)
y_predict = model.predict(Xـtest)
accuracy = accuracy_score(y_true=y_test, y_pred=y_predict)
print(accuracy)
## prints 0.947
print(pd.Series(y_test.reshape(-1)). value counts())
# prints:
#1 949
#0 51
# dtype: int64
The premise is the following: the person was working on a binary classification task. The model’s accuracy during development was high but it didn't perform well in real life. So they think the issue is with the training process. What is it?
I've been looking at it for a while now but I have no idea what to say. The imports are correct, calling the model is correct, fitting and accuracy measurements are correct. The only possible issues I see is that maybe the split is too big? Any thoughts/hints on this?
X test
, there isX_test
$\endgroup$y test
there isy_test
$\endgroup$