- How can I make sure that the initial model trained on a small dataset will not suffer from overfitting before applying the active learning sampling techniques? because I will use this model to select new unlabeled samples.
I'm not totally sure about my answer so please take it with a grain of salt.
I think you shouldn't worry too much about the initial model being overfitted:
- This is likely to happen since the initial dataset is small, so the model might have no choice other than to capture patterns that happen by chance.
- The process of active learning is intended to "correct" the initial model progressively. This is not only about the model capturing new details, it can also be about the model re-evaluating previous patterns based on the data.
So my intuition would be to just let the model overfit a bit if it has to. However, if the model overfits a lot and/or is too complex, it means that it will require a lot (possibly too many) instances to be labeled. Depending on the context, this could be a more serious problem: the initial model should be decent enough for the active learning process not to need the labeling of many/all instances.