I am currently working with a dataset that seems very easily separable and I have an accuracy of 99% for SVM (NN-98%, RF-98%, DT-96-97% and I have checked for leakage & overfitting). As part of my project, I am also learning how to implement a hybrid model but its accuracy is also 99% (1 record misclassified). It seems to be misclassifying the same record as SVM and no other algorithm seems to get that record correct. Probability wise- RF and Logistic Regression give a probability of 0.5-0.7 but all other models give 0.9-1 probability for the wrong prediction. I was wondering what I should do now. Are there certain techniques I can use to reduce the probability to below 0.5 in the stacked model so that it isn't classified the way it currently is? I am using the StackingCVClassifier for implementation and this is a binary classification project.
I am currently using SVM, LR, RF as base models and NB as the meta-model.
My other question is if there is even value in making a hybrid model if the accuracy is the same as SVM's?