Is it true that we assume our P(y|x;theta) to follow Bernoulli's distribution given y has binary output in Logistic Regression? Is there any specific reason why we consider Bernoulli's distribution?
If 1) is true., What happens if we consider P(y|x; theta ) as Gaussian distribution? What would be our cost function? I mean if we assume Gaussian, we can have optimal value of mean and variance by taking mean of (y|x) and its variance which is what we are maximizing the likelihood for.
A Bernoulli distribution is a probability distribution where a random variable takes the value 1 with probability p and the value 0 with probability 1-p. This distribution maps very well onto logistic regression because the random variable is the target variable and the model is estimating the probability of 0 or 1 given the specific values of the features.
The Gaussian distribution is not as well suited because it is a continuous probability distribution that is goes from negative infinity to positive infinity. The result would be predictions outside the bounds of the problem space.
The cost function is something different. The cost function is how errors in prediction are penalized.
