I am using a CNN based model to do sequence classification. Since training an entire dataset is very expensive, and I have a large set of features needed to try, its impossible for me to select features by full trainings.
Usually a good sanity check before training, is try to let model overfit a small set of training samples, to make sure the model is at least capable of remember a small sample size.
Borrowed from this idea, my question is, can I train a small subset of training data, and use its loss curve as a metric, to select best features? Each training is to test how quick would the training loss converges given a subset of selected features.