I have a data set in which the data is coming from various sources. Approx 3k records were verified manually and respective source is tagged if the data comes from that source and is valid/correct.
I am trying to implement a Voting Classifier for this scenario. The basic flow of which is shown below:
Refer a sample of the accuracy and weight table below:
I am directly taking the accuracies as the weights. However, I feel that coverage too has a role to play here. I am unable to figure out a way to formulate the combination of accuracy along with coverage.
For example w.r.t src_3 in the Sources column, the accuracy doesn't seem like a justifiable weight for the source considering the coverage on the Data Set.
How do I go about it?