2
$\begingroup$

I've been watching Dr. G. Hinton lectures on Neural Networks in Machine Learning, and in one of the lectures he explains what the goals of Unsupervised Learning are.

I am having trouble understanding the part where high-dimensional inputs such as images live on or near a low-dimensional manifold (or several such manifolds). What is a manifold exactly, and why is this the case?

Thanks!

machi

$\endgroup$
1
$\begingroup$

Depends who you ask but generally mannifold is just some structure in a (high) dimensional space having finite dimension : a line, a curve, a plane, a rock, a sphere, a ball, a cylinder, a torus, a "blob"... something like this :

enter image description here

If you ask a mathematician they would say that it is a general term describing "a curve" (dimension 1) or "surface" (dimension 2), or a 3D object (dimension 3)... for any possible finite dimension $n$. A one dimensional manifold is simply a curve (line, circle...). A two dimensional manifold is simply a surface (plane, sphere, torus, cylinder...). A three dimensional manifold is a "full object" (ball, full cube, the 3D space around us...).

TO ANSWER YOUR QUESTION what he is referring is that information in this high-dimensional space can be compressed and perserved in some smaller space. (remember images when quantified live in some space that contains information about them)

|improve this answer|||||
$\endgroup$
1
$\begingroup$

A manifold is some kind of low-dimensions structure that exists in a higher-dimensional space. The classic example of this is the Swiss Roll dataset, which simply looks like a spiral with values that vary monotonically along the curves (represented by colors here).

enter image description here.

The overall idea is that there is a simple, 1-dimensional representation of the color values, which can be seen if you unfurl the roll. But if you look in the original space and try to do some kind of nearest-neighbor clustering, you might determine that the dark blue and yellow-green samples are most similar, since they are closest in the original space. Finding the low-dimensional manifold allows you to cluster only along the manifold axis (which is a spiral in this case), giving you the "proper" representation of which samples are close to one another.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.