I have a large dataset of several billion inputs of categorical attributes. The neural network performs a regression task. Since I want to train the model, but not always use all of the data, how can I find a representative subset of the data that is representative for the whole dataset, while capturing the dependencies and generalizing to the other data? Currently I found only approaches for classification and ordinal attributes, where they find centroids as representatives, but for categorical data this is not useful.

  • $\begingroup$ What kind of data are we talking about? Is your data somehow time-dependant? If so, it might be wise to use recent data. Also, what do you mean with "not always use all of the data"? Do you plan on retraining your model from scratch regularly? There is also the option to keep old models and train them further on new data. A good rule of thumb is to use data that is similar in distribution as the data you are trying to forecast. $\endgroup$ Commented Sep 22, 2020 at 17:51
  • $\begingroup$ Does your data have a large number of records or a large number of categorical features or do your categorical features have a large number of values they can take ? $\endgroup$ Commented Sep 23, 2020 at 3:23
  • $\begingroup$ @AnkitaTalwar The data has up to 10 records each taking an enormous number of values. $\endgroup$
    – anascmidt
    Commented Sep 23, 2020 at 7:11
  • $\begingroup$ @NiklasvMoers The data is not dependent. The patterns are graph patterns, from which I want to sample enough useful data for the model to generalize. The retraining is not needed. $\endgroup$
    – anascmidt
    Commented Sep 23, 2020 at 7:12


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.