# Predicting which drug is most appropriate for which patient returns an accuracy of almost 0

I have a dataframe that looks like this:

data = {'age': [54, 21, 7, 18],
'sex': [0, 1, 1, 0],
'disease_type': ['A', 'B', 'A', 'F'],
'change_in_pain': [-0.54, -0.89, 0.07, -0.01],
'drug': ['drug_1', 'drug_7', 'drug_1', 'drug_89'],
}
df = pd.DataFrame(data)


=>

   age  sex disease_type  change_in_pain     drug
0   54    0            A           -0.54   drug_1
1   21    1            B           -0.89   drug_7
2    7    1            A            0.07   drug_1
3   18    0            F           -0.01  drug_19
...


The real df has > 10000 rows (=patients) and 34 different drugs but seemingly I cant upload a csv here for a more usable example?

I would like to train a model that predicts which drug is most effective for which patient given the patient’s age, sex, disease type and how much the pain was reduced (a more negative “change_in_pain” column is better).

In this simple example “drug_1” woud work for disease A only if the patient is older and female.

I wrote the following code but the mean accuracy is returned as almost 0 :

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier

# shuffle
df = df.sample(frac=1.0).reset_index(drop=True)

X = df[['age', 'sex', 'disease_type', 'change_in_pain']]
y = df['drug']

# convert categorical variable into dummy/indicator variables.
X_OHE = pd.get_dummies(X)
y_OHE = pd.get_dummies(y)

X_train, X_test, y_train, y_test = train_test_split(X_OHE, y_OHE, test_size=0.20)

scaler = StandardScaler()
scaler.fit(X_train)

X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)

knn = KNeighborsClassifier(5)
knn.fit(X_train, y_train)
score = knn.score(X_test, y_test)
print('mean accuracy: {:2.2f}'.format(score))


I also tested: RandomForestClassifier(max_depth=5, n_estimators=10, max_features=1), KNeighborsClassifier(3), DecisionTreeClassifier(max_depth=5), MLPClassifier(alpha=1, max_iter=1000) but again the mean acc is around zero.

What am I doing wrong?

EDIT:

Doing it more slowly using:

knn.fit(X_train, y_train)  # X_train: 8000x11, y_train: 8000x34
y_pred = clf.predict(X_test)  # X_test: 2000x11, y_pred: 2000x34
acc = accuracy_score(y_test, y_pred)


shows that y_pred seems to contain only zeros - but why?

• You predict "drug" but you say you want to see "which drug is most effective". Why don't you look at change_in_pain = drug + ... ? Isn't this what means "effectiveness" here? Oct 15, 2020 at 17:32
• ... good point! I mean i want to predict which drug to use but I could re-formulate the question like this: ['age', 'sex', 'disease_type', 'drug'] => 'change_in_pain' and then predict every drug for a patient and take the drug with the largest reduction in pain Oct 16, 2020 at 11:07
• My take is that this is a case for "causal modeling", where you estimate the marginal effect of a drug on pain contolling for age, sex, disease_type etc. This is where I would start. Probably you would need to run different regressions for each disease_type. Depends on the data (and the possible interactions in it which you would need to model somehow). Oct 16, 2020 at 11:13

I can't say exactly why you get null accuracy, but I have some comments that may help:

• you are mixing continous data and categorical data. You might already be aware:
• when standardizing your data: you are although standardizing your categorical data (sex, disease_type), just be sure that it makes sense (depending on the classifier you're using)
• depending on the classifier, mixing continous and binary data may lead to undesirable results. This post explains it in the case of KNN

Otherwise I don't see anything suspicous after a first look. Could you share more data so we can reproduce your result ?

• I could share a csv ... but how can I post it here? Oct 16, 2020 at 11:02
• You can't (see this post). You could try to extract a small amount of representative data that reproduce your issue (that would be the best) or share the full data using a free web platform. The web platform should really be the last try. Oct 16, 2020 at 14:10

You should visualize your data to see what kind of decision bound might fit. Possibly, there is no weighting of features that can predict drug type.