Suppose that in a binary classification task, I have separate classifiers A, B, and C. If I use A alone, I will get a high precision, but low recall. In other words, the number of true positives are very high, but it also incorrectly tags the rest of the labels as False. B, and C have much lower precision, but when used separately, they may (or may not) result in better recall. How can I define an ensemble classifier that gives precedence to classifier A only in cases where it labels the data as True and give more weight to the predictions of other classifiers when A predicts the label as False.

The idea is, A is already outperforming others in catching true positives and I only want to improve the recall without hurting precision.

  • 4
    Ensemble models learn the correct weights for you. Read about boosting and stacking. You can tune the ensemble classifier to yield the recall/precision trade-off you desire. Welcome to the site! – Emre Jan 11 at 19:27
  • can you describe the data? what kind of classifiers you are using? – Bashar Haddad Apr 12 at 0:35

Feature-Weighted Linear Stacking might be what you are looking for.

FWLS combines model predictions linearly using coefficients that are themselves linear functions of meta-features.

In your example you can use the meta-feature "Does A label the example as True?"

based on your description, it looks like different models have different biases. two important questions: do you have any data imbalance problem? what kind of models you are using? using stacking based classifier is beneficial if you have different biases. Try to use a simple stack based classifier. for your level-1 classifier, use different models (e.g. SVM-L, SVM-NL, DT, RF, ... etc). For your meta-data, use probabilities and for the meta-classifier use Random Forest.

if you have data imbalance problem using stack based classifier is a little bit more challenging.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.