Here is my question in my assignment:
You have built a classification model with 90% accuracy but your client is not happy because False Positive rate was very high then what will you do?
This is the question..nothing is given in the background
$\begingroup$
$\endgroup$
3
-
$\begingroup$ You have an imbalanced data set, therefore you would deal with it before building a classifier, as the classifier would inevitably be biased in that case. $\endgroup$– user2974951Commented Sep 25, 2018 at 6:29
-
$\begingroup$ 90% accuracy for an imbalanced data set simply requires the imbalance to be calculated and the accuracy to be adjusted (increased) accordingly. $\endgroup$– M__Commented Jun 5, 2019 at 13:37
-
$\begingroup$ any feedback on my answer? $\endgroup$– Francesco PegoraroCommented Jun 5, 2019 at 15:10
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
3
I think the only general solution would be to: increase the threshold of the model confidence.
For example you are doing binary classification of dog in images: Dog = 1, No Dog = 0
Generally a model (like Neural Network) would output the probability of the image being 1: if it's > 0.5 then predicts 1 else 0. Increasing the confidence to 0.7 would decrease the False Positive.
answered Sep 24, 2018 at 20:40
-
2$\begingroup$ Restating @Francesco’s answer - you can use the ROC curve to select a desired false alarm rate. You’ll have to recode the classification output using this threshold after outputting the probability values. $\endgroup$– HEITZCommented Sep 24, 2018 at 22:13
-
-
$\begingroup$ Do you plan to accept the answer? $\endgroup$ Commented Sep 26, 2018 at 11:21
$\begingroup$
$\endgroup$
1
In my opinion, we should not consider the only accuracy as a performance measure as it evaluates only true positive, true Negative, and the sum total of a model. We have many performance measures like recall, precision, and f1-score. Now, coming to this question statement the classification model with 90% accuracy having a high false-positive rate. First of all False positive rate is a parameter of error metric derived from the confusion matrix. The confusion matrix depends on distinct respective model. Thus, each classification model will have different confusion matrix which turns out to have different False positive rate may be low or high as compared to the previous model. Thus, here we can go for various classification models available like logistics regression, Decision Tree, Neural networks, Random Forest, etc, and check false positive rates using confusion matrix for each of the models. In comparison we can conclude which machine learning model or statistical model is the best fit having high accuracy and lowest possible false-positive rate. A Machine learning paradigm known as ensemble learning can also be used in this condition. Ensemble learning is nothing but the group of different types of machine learning models developed using the same training dataset (some feature may or may not differ in the dataset). Ensemble learning is implemented in a technique known as bagging or Bootstrap Aggregating in which several models are trained on a dataset and the mean of the output is taken for the test dataset output by each model. Random forest is one such ensemble learning technique that aggregates the output of several decision trees to get the most appropriate result.
-
$\begingroup$ christ, you must learn concision! $\endgroup$– mccurcioCommented Aug 4, 2020 at 13:19
$\begingroup$
$\endgroup$
This is likely to be caused by an imbalanced dataset. It means that some observations are less numerous, therefore your model is not learning enough about them. A possible solution might be: use Mini-Batch Gradient Descent optimization, and build the mini-batches in a way that the number of observations is balanced across all the classes. This would attribute greater weight to the observations that are less frequent.
