Linear Regression Loss function for Logistic regression

Question

I was attending Andrew Ng Machine learning course on youtube Lecture 6.4 He says what a cost function will look like if we used Linear Regression loss function (least squares) for logistic regression

I wanted to see such a graph my self and so I tried to plot cost function J with least square loss for a losgistic regression task.

Here is my code

import matplotlib.pyplot as plt
import numpy as np
import math


x = np.random.rand(10000)
# x = np.array([0.1, 0.2, 0.7, 0.4])
y = np.round(np.random.rand(10000))
b = 1

J_list = []
w_list = []
for w in np.arange(-500.5, 500.5, 0.05):
    J = (1/10000)*np.sum((1/2)*np.square(((1/(1 + np.exp(-1*(w*x + b)))) - y)))
    J_list.append(J)
    w_list.append(w)

df = pd.DataFrame()
df['w'] = w_list
df['J'] = J_list

import seaborn as sns
sns.lineplot(x='w', y='J', data=df)

The output of lineplot is

Note: w in my code in theta in Andrew Ng's lecture

If anyone can help me spot my mistake, would be really appreciated.

Answer 1

You have taken x,y from random space. So, it should not follow any rule/logic.

Get a real data e.g. Iris(just 2 class) or using sklearn make_classification module
Figure out the approx value of theta for a good model
Then loop before and after this value
Have y_true in probability(not the Class)