How do I calculate the maximum likelihood (machine learning statistics) of this table of data?

Question

I’m taking a machine learning course and I’m stuck. I've already tried to ask around in the forums of the class but I don't understand how to implement the formulas they give me. Can you please help me understand how to solve this problem from the ground up?

“Calculate the likelihood of this data.”

X | Y

2.5 | +1

0.3 | -1

2.8 | +1

0.5 | +1

the current estimates of the weights are $w_0 = 0$ and $w_1 = 1$. ($w_0$: the intercept, $w_1$: the weight for $x$).

Now Here's what I've already done in case that matters:

When I went on to the forums to try to understand I was told this:

You can use the sigmoid function to find the likelihood of any given one, and then once you have the probabilities of any given one occurring you should be able to find the probability of the whole thing

Which is nice; now I know it has something to do with the sigmoid function: $$\frac{1} {1 + e^{-x}}$$

sigmoid(2.5) = 0.9241418

I hope I'm on the right track, I really have no way of knowing. So then I calculated the sigmoid for all of them and multiplied them together hoping that would give me the probability of the whole thing because that's how you combine probabilities, right? (The probability of a dice role hitting 2 three times is $1/6 \times 1/6 \times 1/6$).

But that apparently isn't the right answer, and I didn't take into account the $y$ column or the weights because I don't know how. How sad for me.

Can you please explain it to me, I feel like I'm a blind man looking for the right colored paper.

Answer 1

You might have mistaken the equation for maximum likelihood , and it should be like this :

$p^y*(1-p)^{1-y}$

while y is your label , it should be 0 or 1 , 0 for negative and 1 for positive , and p is your sigmoid function value.

so , in your case it should be :

p(2.5)*(1 - p(0.3))*p(2.8)*p(0.5)

Hopes this will help you , good luck !

Answer 2

Okay, so here's how it works,

1. $y$, which is the response variable consist of two binary classes, $0$ and $1$. i.e. $y \in \{0, 1\}$
2. Given a threshold value, (we'll use $0.5$ for illustrative purposes), use it to determine the class of $y$ from the logistic or sigmoid classifier output (i.e. $h_\theta(x)$ ). $$\text{If }\text{ } h_\theta(x) \geq 0.5 \text{ <}threshold\text{>}, \text{ predict } "y = 1"$$ $$\text{If }\text{ } h_\theta(x) < 0.5 \text{ <}threshold\text{>}, \text{ predict } "y = 0"$$
3. You will compute a logistic function (also called a sigmoid function) to get the estimated probability that $y = 1$ on input $x$ given by this formula, $$h_\theta(x) =\frac{1}{1 + e^{-\theta^{T}x}}$$ where, $\theta$ are the weights, also known as the parameters of the model. In this case, $w_0$ and $w_1$
4. The weight $w_0$ is for the included bias term of $1$ for all row entries in the dataset.

For each row in the dataset, repeat #5 to #7

5. Compute the value of $\theta^Tx$ as follows, $$\theta^Tx = w_0x_0 + w_1x_1$$ For example, $\theta^Tx$ for the first row will be, $$\theta^Tx = (0 \times 1) + (1 \times 2.5)$$
6. Substitute that value into the sigmoid function $h_\theta(x)$, i.e. sigmoid(x) already defined in #3 above. (P.s. remember the minus in that computation)
7. Test your output, which is a probabilistic value between $0$ and $1$ (i.e. $0 \leq h_\theta(x) \leq 1$) against the threshold value given in the problem. This assigns the output of sigmoid(x) to classes as shown in #2 above.

I hope this helps :)