1
$\begingroup$

Am working on a binary classification using logistic regression data

I have 1000 rows and 28 features. Three to 4 variables are Id variables like product_id, subject_id etc

During train_test split, I drop them like as shown below

X = df.drop(['status','Product_ID','subject_ID'], axis=1)
y = df.status
X_train, X_test, y_train, y_test = train_test_split(X, y, 
                                                    test_size=0.25,
                                                    random_state=0)

Once I do this I do some preprocessing and modelling tasks as below

a) encoding categorical variables for train and test separately b) model.fit() c) model.predict() d) Finally, I get the y_pred and I compare it with y_test.

My question are as follows

a) when there is no identifier in y_pred, how can I link back to get the full row of that instance? Meaning, I want the full data row of that observation along with new column predicted_status (beside already existing actual status column). Is there anyway to include ID variables in model building process but make their effect as 0 (or just stay there as a useless column)

b) Would the same order be preserved during train-test split,encoding,cross-validation,testing etc

c) What happens if we split train and test based on some criteria like year between 2015-2020 (becomes train) and anytime after 2020 becomes test?

$\endgroup$

1 Answer 1

2
$\begingroup$

Keep subject_ID and after train_test_split pass to the model dataframes without the ID column, as in:

df.loc[ : , df.columns != 'subject_ID']

Unless you are explicitly shuffling datapoints during prediction, I believe that commonly returned predictions persist the initial order. Definitely worth checking with the particular model you are using.

EDIT:

See an example of entire process below:

import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.model_selection import train_test_split


iris = sns.load_dataset('iris')
# you don't need this, you have subject IDs.
iris['Subject_ID'] = numpy.random.randint(1, 10000, iris.shape[0])
# use subject IDs as index.
iris.set_index('Subject_ID', inplace=True)

# Split your data, index will be persisted.
Xtrain, Xtest, ytrain, ytest = train_test_split(
               iris.loc[ : , iris.columns != 'species'], 
               iris['species'],
               random_state=1
               )

# Model and predict.
from sklearn.naive_bayes import GaussianNB  # 1. choose model class
model = GaussianNB()                        # 2. instantiate model
model.fit(Xtrain, ytrain)                   # 3. fit model to data
y_hat = model.predict(Xtest, )              # 4. predict on new data

# Append predictions, original datapoint IDs will be encoded in the index.
Xtest["predictions"] = y_hat

# Test by joining original labels, using a dummy dataset for prediction.
Xtest = Xtest.join(iris['species'], how='inner')
Xtest.head()
>>>
Subject_ID sepal_length sepal_width petal_length petal_width predictions species
2265 5.8 4.0 1.2 0.2 setosa setosa
1961 5.1 2.5 3.0 1.1 versicolor versicolor
4177 6.6 3.0 4.4 1.4 versicolor versicolor
6041 5.4 3.9 1.3 0.4 setosa setosa
8500 7.9 3.8 6.4 2.0 virginica virginica
$\endgroup$
11
  • 1
    $\begingroup$ thanks, upvoted. BUt after prediction, how do I link back? $\endgroup$
    – The Great
    Jan 28, 2022 at 12:48
  • $\begingroup$ See example, hope it helps. $\endgroup$
    – hH1sG0n3
    Jan 28, 2022 at 13:34
  • $\begingroup$ Hi, one question on this. I know we usually predict on test set. But we can also predict on train set and link their prediction back to the train data points. Am I right? While I understand it doesn't make sense to train and predict using the same dataset. But to verify the confusion matrix of train set, I have to predict on train set; Am I right? $\endgroup$
    – The Great
    Feb 4, 2022 at 7:50
  • $\begingroup$ Of course, test set prediction is usually considered for performance of model but is it okay to build confusion matrix, precision recall etc for train set as well? $\endgroup$
    – The Great
    Feb 4, 2022 at 7:50
  • $\begingroup$ In the above code, you have not excluded subject_id column from being passed to the model? However, your text says we should remove it from being fed to the model. Sorry, if I understood this incorrectly. can help me with this? $\endgroup$
    – The Great
    Feb 4, 2022 at 8:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.