# What is the significance of underflow during parameter update using stochastic gradient descent?

### Background

I am using scikit-learn's MLPRegressor to learn a model with the following arguments:

hidden_layer_sizes=(16, 16),
learning_rate_init=1e-2,
solver='sgd',
momentum=0.9,
activation='tanh',
learning_rate='constant'


I use partial_fit() to incrementally fit the model. Occasionally I get a FloatingPointError on this line in the MLPRegressor's code:

updates = [self.momentum * velocity - self.learning_rate * grad

>>> FloatingPointError: underflow encountered in multiply


### Problem

I am getting an underflow error when the model is calculating how much to update parameters by multiplying learning rate, momentum, velocity, and error gradients.

Does not an underflow mean that the updates are too small? Does it imply the model has reached some local minimum? Should I take this as a hint that:

• I have exhausted my data and training has ended,
• I should up my learning rate to keep updates happening
• Something else?