# Model Performance using Precision as evaluation metric

I am dealing with an imbalanced class with the following distribution : (Total dataset size : 10763 X 20)

0 : 91%

1 : 9%

To build model on this dataset having class imbalance, I have compared results using

1) SMOTE and

2) Assigning more weight to the minority class when applying fit

and the latter seems to be working better.

After experimenting with Decision Tree, LR, RF SVM(poly and rbf) I am now using XGBoost classifier which gives me the below classification results(these are the best numbers I've got so far)

The business problem I'm trying to solve requires the model to have high precision, as the cost associated with that is high.

Here's my XGBClassifer's code :

xgb3 = XGBClassifier(
learning_rate =0.01,
n_estimators=2000,
max_depth=15,
min_child_weight=6,
gamma=0.4,
subsample=0.8,
colsample_bytree=0.8,
reg_alpha=0.005,
objective= 'binary:logistic',
scale_pos_weight=10,
eval_metrics = 'logloss',
seed=27)
model_fit(xgb3,X_train,y_train,X_test,y_test)


And here's the code for model_fit :

def model_fit(algorithm,X_train,y_train,X_test,y_test,cv_folds = 5 ,useTrainCV = "True", early_stopping_rounds = 50):
if useTrainCV:
xgb_param = algorithm.get_xgb_params()
xgbtrain = xgb.DMatrix(X_train,label = y_train)
xgbtest = xgb.DMatrix(X_test)
cvresult = xgb.cv(xgb_param,xgbtrain,num_boost_round = algorithm.get_params()['n_estimators'],
nfold = cv_folds,metrics='logloss', early_stopping_rounds=early_stopping_rounds)
algorithm.set_params(n_estimators=cvresult.shape[0])

algorithm.fit(X_train, y_train, eval_metric='logloss')
y_pred = algorithm.predict(X_test)
cm = confusion_matrix(y_test,y_pred)
print(cm)
print(classification_report(y_test,y_pred))


Can anyone tell me how can I increase the precision of the model. I've tried everything I know of. I'll really appreciate any help here.

• What is the type of your data? is it images? – Hunar Mar 22 '19 at 17:27
• No it is mostly numerical Healthcare data with attributes like age, gender, smoke (yes/no) and the tests a person has taken. All the pre processing steps have been performed. – Shekhar Tanwar Mar 22 '19 at 17:31