# Combining results from classifiers trained on different test/train splits results in higher accuracy

I have developed a classifier model using LightGBM. The accuracy of the model varies significantly because of the test_train_split state(between 83% and 91%). This is normal due to the nature of the data.

I have noticed that using 3 models - trained and tested on the same data, but using different random states - and combining their results gives a higher overall accuracy.

For example: Model 1 will classify Sample 1 as Category A. This result would be wrong.

However, running the three models (trained on different subsets of the data), will give the following results:
Model 1: Category A,
Model 2: Category B,
Model 3: Category B.
Category B is indeed correct.

Is this a good practice? It seems quite untraditional, but it is working. Is there a better way to get the same results? I have thought about using all data to train the model and not splitting it, however LightGBM needs a test set as far as I understand.

5. It is vital to have data in multiple independent buckets (for training/validation/testing). Training on all the data will defintely give you better results in accuracy but will fail miserably in production. It is referred to as overfitting.