If I get 95+ % accuracy in normal models, should I still consider Ensemble models? Why should I choose Ensemble models over normal models?
Firstly, welcome to the site!
When do we use Ensemble model?
when there are 2 models which perform moderately then we combine their results to get a model which performs better. In your scenario you already have a model which gives you good results what is the point of implementing Ensemble Models?
As @Tagoma said, it depends on your data and your goal. For example, you are trying to predict stock rates, every 0.01% matters. In such scenarios you need to use complex algorithms to maintain balance on that slim line i.e., not to over-fit, not to over-train and just predict.
Measures to check if your model is over-trained is by giving some random data and see how it performs, add some noise in the training data.
One more important thing to do is, to check for the Predictor Importance and see if there is any highly correlated feature with the target variable. For example, you are planning to predict age and if you have DOB as a feature in that yes of course you would predict with 99.99% accuracy but that is not what we use ML for.
If all of them are satisfied and achieving that accuracy then it means that your model's performance is good.
Finally, Implementation of Ensemble is dependent on your business problem and your understanding on business.
It is a problem specific and data specific problem. For one problem, 95% accuracy may be good, for some other it might not be very good and there might still be room for improvements. Likewise, it might happen that the problem is well defined, your architecture is good but due to a lot of noise in the data, you can't do better than 95%. In that case, we will say the results are good.
Ensembles: Ensembling is a general term for combining many classifiers by averaging or voting. It is a form of meta-learning in that it focuses on how to merge results of arbitrary underlying classifiers. Generally, ensembling improves the final results but again, not necessarily. If you are ensembling two highly correlated models, your ensembled model will give almost the same result as your stand-alone models.
The answer to your question is only possible through data analysis and the task.
1. Data Analysis: If you are trying to classify data with 38 samples as A and 2 samples as B, a Majority Vote Classifier (MVC) might be good enough to get 95% accuracy too. That's where you might want to check balanced accuracy or F-1 scores. Here, the MVC may give you 95% accuracy like your normal models, but its F-1 score will be 0.
2. Task: If you are trying to predict multi-class labels and are getting 95% accuracy, your results might be good. But if your task is like something described here, the model could be over-fitting and you might be missing the key output you are looking for, i.e. accurately predict the rare example.
I would suggest determining the loss function might be very useful. Ensemble methods usually help if they are trying to complement each other (as you might have seen in various Kaggle competitions). Also, you can look at Gradient Boosted Trees and XGBoost which create multiple small decision trees, with each new tree trying to predict the previous tree's incorrect result.