Logitic Regression cost function - what if ln(0)?

Question

I am building logistic regression from scrap.

The simplified cost function I am using is (from machine learning course on coursera):

in specific case during learning, one observation in training set y is 0 - but the specific choice of betas in:

makes g(z) = h(x) = 1 , because e.g. z > 50.

in this case my right side od J is (1 - 0) * log(1 - 1) what is -inf (I am doing my calculations in python)

I understand that in this case value of cost function should be high because the probability of y = 1 is very big while the truth is that it actually is 0.

Is the problem approximation of g(50) being 1 instead of something like: 0.999999? Or there is some more fundametal error with my logic?

because this specific example the summation of cost of all observations is nan (not a number) in my code.

Answer 1

In practice, an offset is used to avoid log explosion due to values close to zero. For example $\hat{\text{log}}(x)=\text{log}(x + \text{1e-6})$.