I have a text-classification problem with a lot of training data. Running cross-validation takes a lot of time - several days or even weeks. In order to make the system more responsive, I am thinking of the following scheme:
- Train a network on instances 1,...,100.
- Test it on instances 101,...200; output the accuracy.
- Train the existing network on instances 101,...200;
- Test it on instances 201,...300; output the accuracy.
- And so on.
Ideally, I would like that:
- Training on each additional 100 instances will take a constant time (i.e, I will not have to re-train the network on all previous instances);
- The trained network will have the combined "wisdom" of all previous instances (so its accuracy will tend to improve with time).
Is this possible to do with standard deep-learning tools (e.g. dynet)?