1
$\begingroup$

How to extract true positives data from testing data after training and testing?

For example in test data I've two rows and one row is true positive and another is false negative. But i want rows which has only true positive data. How to extract that complete row from testing data after training and testing?

$\endgroup$
0
$\begingroup$

One thing you can do is browse your prediction vector, get the indexes of "1" responses, and then check those indexes in y_test. if your y_test[index] is also a "1" class, then select the row by index in X_test

I tested this, it works for me. In my case, my X and y are pandas.DataFrame.

    import pandas as pd
    from sklearn.linear_model import LogisticRegression
    import numpy as np
    
    X_train = pd.read_csv("saves/cv_sets/X_train1.csv", sep=";", encoding="latin1")
    X_test = pd.read_csv("saves/cv_sets/X_test1.csv", sep=";", encoding="latin1")
    y_train = pd.read_csv("saves/cv_sets/y_train1.csv", sep=";", encoding="latin1")
    y_test = pd.read_csv("saves/cv_sets/y_test1.csv", sep=";", encoding="latin1")
    
    clf = LogisticRegression(class_weight="balanced", solver='lbfgs', C=0.1)
    model = clf.fit(X_train, y_train)
    
    pred = model.predict(X_test)
    
    pred1 = np.where(pred==1)
    
    TP_Indexes = []
    for k in pred1[0]:
        if(y_test.iloc[k][0] == 1):
            TP_Indexes.append(k)
    
    
    X_test_TP = X_test.iloc[TP_Indexes]
| improve this answer | |
$\endgroup$
  • $\begingroup$ Can you please write code for that I'm confused $\endgroup$ – Nithin Reddy Jul 31 at 9:12
  • $\begingroup$ "pred1 = np.where(pred==1)" --> This line is for selecting the lines your model predicted as 1, so if you want fn or tn, change to pred==0. "y_test.iloc[k][0] == 1" --> This line is for selecting lines on your dataset that are real 1, so if you want fp or tn, change it to == 0. (also, accepting the answer would be appreciated ;) ) $\endgroup$ – BeamsAdept Jul 31 at 10:15
  • $\begingroup$ Thanks man and do you know how to assign back categorical variables to main data after training and testing? (using Inverse_transform) $\endgroup$ – Nithin Reddy Jul 31 at 10:46
0
$\begingroup$

If you use label binarization function from scikit learn for encoding the labels before training then it has a built in inverse_transform function Please go through this link /https://scikit-learn.org/stable/modules/preprocessing_targets.html/

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.