Most problem with machine learning projects I have faced is the lack of data. The samples available are enough to disqualify rule based approach but not enough for a neural network to train.
For example, to train a neural network (or even fine tune a pretrained model) on a new entity in an NER system takes a thousands of different records. And the requirement of these thousands of records is to have enough variations to avoid overfitting.
Generally as human we can detect patterns by carefully observing the data, however it becomes humanly not possible to detect all patterns in inputs, and that is where deep learning comes into play of automatically detecting the patterns to make a hypothesis.
Now my question is, what are possible ways, using which limited data can be used to train a neural network with limited data. Let me add some inputs from my side, which I believe are not sufficient:
- Data Augmentation : for images rotating, scaling and skewing. For textual data, repeating text with some masking and embedding/synonym replacement.
- what else?