1
$\begingroup$

I was attending Andrew Ng Machine learning course on youtube Lecture 6.4 He says what a cost function will look like if we used Linear Regression loss function (least squares) for logistic regression Lecture slide

I wanted to see such a graph my self and so I tried to plot cost function J with least square loss for a losgistic regression task.

Here is my code

import matplotlib.pyplot as plt
import numpy as np
import math


x = np.random.rand(10000)
# x = np.array([0.1, 0.2, 0.7, 0.4])
y = np.round(np.random.rand(10000))
b = 1

J_list = []
w_list = []
for w in np.arange(-500.5, 500.5, 0.05):
    J = (1/10000)*np.sum((1/2)*np.square(((1/(1 + np.exp(-1*(w*x + b)))) - y)))
    J_list.append(J)
    w_list.append(w)

df = pd.DataFrame()
df['w'] = w_list
df['J'] = J_list

import seaborn as sns
sns.lineplot(x='w', y='J', data=df)

The output of lineplot is

cost function with least square loss in logistic regression

Note: w in my code in theta in Andrew Ng's lecture

If anyone can help me spot my mistake, would be really appreciated.

$\endgroup$
1
$\begingroup$

You have taken x,y from random space. So, it should not follow any rule/logic.

Get a real data e.g. Iris(just 2 class) or using sklearn make_classification module
Figure out the approx value of theta for a good model
Then loop before and after this value
Have y_true in probability(not the Class)

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yes you reason is correct. But 'Have y_true in probability(not the Class)' you just meant to convert a Binary label in 0 and 1 right ? $\endgroup$ – Hitesh Somani Jul 10 at 7:15
  • $\begingroup$ I meant to use the probability output of the model, not the Class. Till probability calculation only, we have any maths functions involved. So, it will depict the underlying variance accurately. Post that we basically do a Floor/Ceiling on the output based on a good Thresold e.g. 0.6 --> 1.0. This should be done only at the end. $\endgroup$ – 10xAI Jul 10 at 7:32
  • 1
    $\begingroup$ yes. The output of the models are in probability only. $\endgroup$ – Hitesh Somani Jul 10 at 7:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.