# Batch-mode active learning: How to select the batch that will bring the largest model improvement

I am facing a problem where I want to use active learning to improve my classifier. Basically, I can choose data from one (and only one) data set among a set of candidate data sets. The question is which one to choose?

In other words, given a set of candidate data sets that I can use improve my classification model, which one is going to improve the model most? Can I use the some metric (eg., average) from the objective functions of the batches? Do I need to normalise the objectives across all data sets? Have metrics to infer the magnitude of model improvement been proposed?

At this stage, my objective function looks like:

a  Uncertainty + (1-a) Diversity


where a is a weight factor, Uncertainty is the uncertainty in the model prediction for a given data point and Diversity is a measure of distance between the data point and the training population.

Any help would be greatly appreciated.

Cheers

Franz