Newbie: What is the difference between hypothesis class and models?

Question

I am new to machine learning and I am confused with the terminology. Thus far, I used to view a hypothesis class as different instance of hypothesis function... Example: If we are talking about linear classification then different lines characterized by different weights would together form the hypothesis class.

Is my understanding correct or can a hypothesis class represent anything which could approximate the target function? For instance, can a linear or quadratic function that approximates the target function together form a single hypothesis class or both are from different hypothesis classes?

Answer 1

Your hypothesis class consists of all possible hypotheses that you are searching over, regardless of their form. For convenience's sake, the hypothesis class is usually constrained to be only one type of function or model at a time, since learning methods typically only work on one type at a time. This doesn't have to be the case, though:

1. Hypothesis classes don't have to consist of only one type of function. If you're searching over all linear, quadratic, and exponential functions, then those are what your combined hypothesis class contains.
2. Hypothesis classes also don't have to consist of only simple functions. If you manage to search over all piecewise-$\tanh^2$ functions, then those functions are what your hypothesis class includes.

The big tradeoff is that the larger your hypothesis class, the better the best hypothesis models the underlying true function, but the harder it is to find that best hypothesis. This is related to the bias–variance tradeoff.