# What is the difference between all the different types of learning within machine learning?

This is a question that is really hard to google, and the differences are confusing. Does anyone have good examples of the differences between them all?

• Supervised Learning
• Semi-Supervised Learning
• Distant Supervision
• Active Learning
• Lightly Supervised Learning
• Weakly Supervised Learning
• Competitive Learning
• Reinforcement Learning
• Unsupervised Learning

Supervised Learning

In general, supervised learning refers to a situation in which you have some $$X$$ that is related to $$y$$, so that you can model how both are related (on average). In this case you can build a statistical model:

$$y = \beta X + u.$$

The model learns $$\beta$$, so that you are able to predict $$y$$ by: $$\hat{y}=\hat{\beta}X$$.

Examples are regression or classification tasks (e.g. using linear models, neural nets, boosting etc).

Unsupervised Learning

When you have only $$X$$ and no response $$y$$, you face an unsupervised problem. In this case, the focus is to identify patterns in $$X$$, e.g. by clustering. Examples for unsupervised learning are k-nearest neighbors (KNN) or principle components (PCA).

The book Introduction to Statistical Learning provides a very good overview. Alternatively, the book Elements of Statistical Learning provides a more comprehensive overview.

Reinforcement Learning

Reinforcement learning is a little different. Here you try to find an optimal response to some problem, based on past experiance. Examples are self driving cars or a game of tic tac toe. So for every state of the environment (e.g. the state of the tic tac toe game), RF aims at finding an optimal response (best next move). Googles DeepMind and AlphaGo are RF-based. The book by Sutton and Barto is the standard literature here.