# How to compute the maximum likelihood hypothesis?

The Bayes theorem states that:

$$\begin{equation} P(h|D) = \frac{P(D|h)P(h)}{P(D)} \end{equation}$$ where $$D$$ is the dataset and $$h$$ is an hypothesis from the hypothesis space $$H$$. Now (I'm not sure so if I'm wrong please correct me) I can consider:

• $$P(h|D)$$ = the probability $$h$$ has generated the dataset $$D$$. More specifically, for each $$h$$ we have a probability that it has generated the dataset $$D$$.
• $$P(D|h)$$ = the probability that $$D$$ has been generated by $$h$$. More specifically, for each possible dataset $$D$$, a certain hypothesis $$h$$ (that we have) can have generated it.

And I can represent them visually, for example:  Now, if we know the prior probability $$P(h)$$ then we can compute the maximum a posteriori hypothesis with the following formula:

$$\begin{equation} h_{MAP} = argmax_{h \in H} P(h|D) = argmax_{h \in H} \frac{P(D|h)P(h)}{P(D)} \end{equation}$$

Otherwise, we can consider the maximum likelihood hypothesis:

$$\begin{equation} h_{ML} = argmax_{h \in H} P(D|h) \end{equation}$$

At this step I don't understand how I compute $$h_{ML}$$ because if I consider $$P(D|h)$$ represented as in the previous example in the cartesian space we have $$D$$ in the x-axis, so if I consider the $$argmax P(D|h)$$ I will find the best $$D$$ and not the best hypothesis $$h$$.

What am I doing wrong? Are probabilities $$P(h|D)$$ and $$P(D|h)$$ not well interpreted in the cartesian space?

We can define a function $$f(D,h) =P(D|h)$$, that is it is a function of both $$h$$ and $$D$$ and we want to maximize it.
In the event that $$D$$ is already fixed to be $$\hat{D}$$, then the goal should be to maximize the function $$g(h) = f(\hat{D}, h) = P(\hat{D}|h).$$
We can plot the function $$g$$, here $$g$$ is a function of $$h$$ rather than $$D$$.
• So you are saying "Since I know the dataset (and I define it as $\hat{D}$, I can compute $P(\hat{D}|h)$ for each hypothesis $h \in H$, after that I return the argument $h$ which maximize $P(\hat{D}|h)$", right? – alex_the_great Oct 29 '18 at 10:32
• yup, that is the meaning of $\arg \max_{h \in H} P(D|h)$. – Siong Thye Goh Oct 29 '18 at 11:47
• ok but in this way I need to know all possible $h \in H$ and in practical problems this is not feasible, so how actually is computed the best hypothesis $h$? – alex_the_great Oct 29 '18 at 12:06
• It depends on your hypothesis space. It can be finite or $h$ can depends on certain parameters that we try to choose to maximize the likelihood. – Siong Thye Goh Oct 29 '18 at 12:14
• parameters you are referring to with respect to $h$ are those defined with the vector $\theta$ or the weight vector $w$? If yes, those weights are for example used in machine learning models like neural networks? – alex_the_great Oct 29 '18 at 12:52