Machine Learning - Range of Hypothesis space and choiceof Hypothesis function type

Question

I am new to machine learning and seek your help in clarifying my elementary doubts. I did a fair amount of googling, but find most literature jumping directly into math.

What I know is that given a labelled training data, a ML algorithm chooses from a hypothesis space H a hypothesis function h.

As an example, assume that a feature vector in training data contains 3 features (x1 through x3)

Now the data from the training set is taken and plugged into a formula (function type). If x is the feature vector and w represents the coefficients of the formula, then output y = A pre-determined f(w,x).

My questions are:

1. Who decides the range of each of the coefficients?   
2. Who decides the formula? Is the formula fixed for a ML algorithm?
3. What exactly is the hypothesis space? Is it range of w, or is it different formulas or both?

I acknowledge I have asked more than one question and against the rules, but it was convenient to logically group them in a single post.

Answer 1

I will answer your question as I understand them, the clarity you are looking for comes from the Maths behind ML.

1. There is no fix range of values of the coefficients or weights. Finding the value of these weights is the work of ML algorithms. although using regularization you can reduce the value of these weights but range depends on algorithm. no. of weights you can decide

2. Yes, formula is loosely fixed for a ML algorithm. e.g. for linear regression in your case

   h(w,x) = w0*x0 + w1*x1 + w2*x2 +w3*x3 (where x0 =1)

the weights will change after each iteration though. for neural network it is multiplication of a weight matrix with an input matrix for each layer. The output of h(w,x) is the prediction of ML which you can compare against y.

3. All the ML algorithms available are your hypotheses space(H)

e.g., quadratic,

linear equation(the one above),

high degree polynomial,

complex neural network.