# When does Adam update its weights?

I have a dataset with at least 70% of labels incorrect.

I'd expect that incorrect labels would compensate each other while true labels will be taught properly (given a very large dataset).

For example, if I have 300 samples saying a => -1 and 300 samples saying a => 1, the result for the input "a" eventually will be 0 (for a regression problem).

If I use Adam for the example above, won't it affect the results for the inputs with noisy labels due to its adaptive nature? Won't it be better to use SGD instead and decay the learning rate, or does Adam change its weights only at the end of every epoch?

• Why are so sure of label compensating each order without breaking the order. What is the cause of incorrect labeling? – 10xAI Jul 24 at 9:40