@Alexey Grigorev already gave a very good answer, however I think that it could be helpful to add two things:
- I would like to provide you with an example that helped me understand the significance of the manifold intuitively.
- Elaborating on that, I would like to clarify the "resembling of Euclidian space" a little bit.
Imagine we would work on a collection of (black and white) HDready images (1280 * 720 pixels). Those pictures live in a 921,600 dimensional world; Every picture is defined by individual values of pixels.
Now imagine that we would construct these images by filling in each pixel in sequence by rolling a 256-sided die.
The resulting image would probably look a little something like this:
Not very interesting, but we could keep doing that until we hit something we would like to keep. Very tiring but we could automate this in a few lines of Python.
If the space of meaningful (let alone realistic) images would even be remotely as large as the entire feature space, we would soon see something interesting. Maybe we would see a baby picture of you or a news article from an alternative timeline. Hey, how about we add a time component, and we could even get lucky and generate Back to th Future with an alternative ending
In fact we used to have machines that would do exactly this: Old TV's that weren't tuned right. Now I remember seeing those and never have I ever seen anything that even had any structure.
Why does this happen? Well: Images we find interesting are in fact high resolution projections of phenomena and they are governed by things that are much less high dimensional. For instance: Brightness of the scene, which is close to a one dimensional phenomenon, dominates almost a million dimensions in this case.
This means that there is a subspace (the manifold), in this case (but not not per definition) controlled by hidden variables, that contains the instances of interest to us
Local Euclidian behaviour
Euclidian behaviour means that behaviour has geometrical properties. In the case of the brightness that is very obvious: If you increase it along "it's axis" the resulting pictures become continuously brighter.
But this is where it get's interesting: That Euclidian behaviour also works on more abstract dimensions in our Manifold space. Consider this example out of Deep Learning by Goodfellow, Bengio and Courville
Left: The 2-D map of the Frey faces manifold. One dimension that has been discovered (horizontal) mostly corresponds to a rotation of the face, while the other (vertical) corresponds to the emotional expression. Right: The 2-D map of theMNIST manifold
One reason why deep learning is so successful in application involving images is because it incorporates a very efficient form of manifold learning. Which is one of the reasons why it is applicable to image recognition, and compression, as well as image manipulation.