What does the repeated message "No further splits with positive gain, best gain: -inf" mean?

Question

I am training a LightGBM classifier on a binary classification problem.

From time to time I get the following message repeatedly:

[LightGBM] [Warning] No further splits with positive gain, best gain: -inf

It almost seems, that if it happens, the training is stuck in an infinite loop Does someone know, what the reason is for this and if it is indeed an infinite loop or if for example this message is output each time the split of a tree node does not succeed and the algorithm continues with the next node letting the current one be a terminal (leaf) node. In the latter case I would maybe wait a bit longer before I terminate the process.

Unfortunately at present I cannot really say with which parameters the training was started since it is performed in a hyperopt loop, I would need to terminate it first.

The training parameters were:

{'n_estimators': 1730, 'max_depth': -1, 'num_leaves': 24579, 'min_child_samples': 450, 'reg_lambda': 0.0, 'silent': False, 'reg_alpha': 0.0}

It seems that for the unlimited max_depth in combination with the high number of num_leaves the algorithm frequently came to a point where it could not find a split with at least 450 samples (the total number of samples is less than 500'000).

Answer 1

I don't know the internal specifics about LGB behavior in this case but I don't think it is stuck in an infinite loop.

From Github's LightGBM issues we can find that:

What's the meaning of "No further splits with positive gain, best gain: -inf" message?

It means the learning of tree in current iteration should be stop, due to cannot split any more. I think this is caused by "min_data_in_leaf":1000, you can set it to a smaller value.

That is, LGB is trying to split the data in leaf but it can't. The greater min_data_in_leaf is the more conservative the algorithm is. Its optimal value depends on the number of training samples and num_leaves.