Improve Precision of a binary classifier - Decision Tree in Python

Question

Currently, I am working on a project. The dataset is balanced roughly in the ratio of 50:50. I created a decision tree classifier. I am achieving decent accuracy (~75%) on validation data but the precision for the target variable is biased. For class=0 it is approx. 98% while for the class = 1 it is just 17%.

I have tried scaling the data using MinMaxScaler still no luck.

model = tree.DecisionTreeClassifier(class_weight={1:30}, min_samples_leaf=160, max_depth=10)

X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size=0.3, random_state=10)

min_max_scaler = preprocessing.MinMaxScaler()
X_train_scaled = min_max_scaler.fit_transform(X_train)
X_test_scaled = min_max_scaler.fit_transform(X_test)

model = model.fit(X_train_scaled, y_train)

prediction = model.predict(X_test_scaled)

print metrics.accuracy_score(y_test, prediction)
print classification_report(y_test, prediction)

Size of x_train_scaled = 12600 and x_test_scaled = 5400
Accuracy: 75%
Precision: {0:100%, 1:17%}
Recall: {0:74%, 1:100%}
F1-Score: {0:85%, 1:29%}

How can I improve the precision of class=1 while still maintaining the overall precision and accuracy?

Answer 1

I guess differences in accuracies between class 0 and class 1 come from the class_weight parameter you have used. Class 1 will benefit from this overweighting towards class 0. You could try to play on this parameter to re-balance your results in class 0 and class 1.

An other idea could be to play on probabilities outputs and decision boundary threshold. Remember than when calling for method .predict(), sklearn decision tree will compare outputed probability with threshold 0.5. If it is greater than 0.5, then it assign class 1. On the contrary, if it is less than 0.5, it will assign class 0. You could try to play on this threshold by outputing probabilities first with .predict_proba()[:,1] and then test results for different thresholds decision boundaries. You can operate such as below :

model = clf.fit(df[features], df[label])
df["proba"] = model.predict_proba(df[features])[:,1]
threshold = 0.4 # You can play on this value (default is 0.5)
df["pred"] = df["proba"].apply(lambda el: 1.0 if el >= threshold else 0.0)

Answer 2

While adjusting the Probability threshold, care must be taken that we use the predictions on the train data to do so. If we adjust the threshold based on the predictions made on the test data, we will be simply overfitting the model on the test data.