I'm new to datascience and appreciate your sage advice! I need to build an incremental learning model, and I know there's a lot that goes into something like that, but I'd like to highlight the most fundamental, abstract requirement I have in my particular case and ask you to focus your attention on that.

However, I build the incremental learning model it must never forget what it has learned. That is when it learns something new it can't forget something it already learned.

I need to know what algorithm or series of algorithms I can use for that.

I'd like to give a toy example to make sure I'm understood. Imagine I decide to use a neural net as the model that learns incrementally. Imagine, as a toy example, that I train it the following 3 inputs → output observations:

0 → 0

2 → 2

4 → 4

We can even visualize this as a 2D graph (y = x): the identity function

We train it, so it has 100% accuracy, great. So we deploy it and now it starts seeing more observations and must learn from them incrementally.

In this example, pretend that when it sees a new input it'll ask me if the model output was correct. If it is correct I want it to remember that specific observation, and never change it, it's fixed. This would mean if it is not correct, all the fixed points in the model do not move.

For instance, let's imagine it see a new input: 1, and comes up with a correct model output: 1. It's memory visualized looks like this: (notice there is a dot at x=1, y=1) still the identity function

So let's imagine it sees a new input: 3, and comes up with a model output: 3, and asks me if it was correct and I say, "nope, the output should be 0." Now it has to do some training to learn that new answer, but as I said I don't want it to mess up anything it's already learned.

As I understand typical neural net training, they become more accurate on average, meaning when you train it on a new observation it might mess up the weights and biases of already learned observations, but on average, fewer and fewer.

That is not what I want. I want to incorporate the new information into the model without messing up verified observations. Now, some of you might say, "dude, that's just a lookup table." But no, it's not, because I still want the new observation to have an effect on the un-trained, or un-"verified" observations (like 2.5 for example) like this: (notice how x=0 is still y=0, and x=2 is still y=2, and x=4 is still y=4, and x=1 is still y=1, but everything else moved?) not the identity function

Alright, so, I hope I've communicated the requirement effectively, my question is: which algorithm do I use to accomplish this model requirement?

The input data and output data of the model will be more complicated than small integers, but I could conform all inputs to be numerical before it gets to the model. I kinda wonder if the boundary to the data mapping (which is the model) can't be directly derived in an N-D space in exactly the same way it is derived on a 2-d grid in the above toy example. Is there a name for doing that?

Perhaps using something as flexible and sophisticated as a neural net to derive that boundary is not actually necessary, idk, I'd like your opinion.

Of course, when you start talking about algorithms, they come with their own requirements for data structures so, I'm not so much asking about algorithms as I am, just asking about how to do this in general. Like best practices, or design patterns, or something.

I'd really appreciate your feedback, expertise, and experience on this problem. Thanks!


1 Answer 1


One possible way to approach training on additional data while maintaining performance on previous data is to have unit tests on model performance. If the updated model weights do not pass the unit tests, then do not accept the updated model weights.

If there is reduced performance on previous data points, those data points can be passed into the model again so updated model weights are encouraged to fit to important previous data points also.

  • $\begingroup$ Though I appreciate the suggestion, it seems like a work around, rather than a pure solution. You're essentially adding on a hack after I've made a new model to see if it's bad or good, I'd rather just not make a bad model in the first place. I'm capable of coming up with hacks, but I'm not smart enough to come up with the pure solution. not for the general case anyway. $\endgroup$
    – MetaStack
    Apr 13, 2022 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.