Please help answer this question or point me to any resource.
There is a model in an environment where training happens with new data and the data is discarded after training is completed. This keeps on happening in cycles. Hence we dealing with models allowing "partial fit"
In this model, there is a training variable X , which is a categorical variable. X has some previously known categories C and its encoded as "one hot vector" of length D .
Now suppose we are now observing a new category of X, so we need to extend encoding of C to length D+1 .
The question is, how to do this without losing all the previously learned knowledge by model.
Training again is not an option since we have discarded all the previously held data and only new data can be used for training.
There is already a question Updating One-Hot Encoding to account for new categories However the answer is either to use entire data to create the "one hot encoding" (which is not possible in this environment since new data is in future) , or to omit the usage entirely (thus loosing some information).