Which learning tasks do brains use to train themselves to see?

Question

In computer vision is very common to use supervised tasks, where datasets have to be manually annotated by humans. Some examples are object classification (class labels), detection (bounding boxes) and segmentation (pixel-level masks). These datasets are essentially pairs of inputs-outputs which are used to train Convolutional Neural Networks to learn the mapping from inputs to outputs, via gradient descent optimization. But animals don't need anybody to show them bounding boxes or masks on top of things in order for them to learn to detect objects and make sense of the visual world around them. This leads me to think that brains must be performing some sort of self-supervision to train themselves to see.

What does current research say about the learning paradigm used by brains to achieve such an outstanding level of visual competence? Which tasks do brains use to train themselves to be so good at processing visual information and making sense of the visual world around them? Or said in other words: how does the brain manage to train its neural networks without having access to manually annotated datasets like ImageNet, COCO, etc. (i.e. how does the brain manage to generate its own training examples)? Finally, can we apply these insights in computer vision?

Answer 1

Maybe this paper will give you an overview about (and entrance to) this topic from biological side. It is a review about the state of the art in human brain development (and its implications for clinical treatment).

The table of contents include for example

• Stage 1: the first year, early maturation of vision and the structure of V1 neurobiology
• Stage 2: preschool children have high variability in V1 development (1–4 years)
• Stage 3: experience-dependent visual development in school aged children (5–11 years)

(V1 means "visual cortex") Where this points all handle 3 categories: visual milestones (i.e. contrast sensitivity, contour integration), anatomical milestones (i.e. morphology) and neurobiological milestones (i.e. synapsis, but also a lot of genetics).

And as second : Maybe you could ask this question on bioinformatics.SE too, because this connection between biological example and computational reproduction is one of their fields.

Answer 2

I think this kind of question is better fit for the Artitifical Inteligence SE, but it works here as well (I guess).

So Natural Neural Networks had a lot of time to develop using Genetic Algorithms (evolution). Even the complex human eye might have started with bacteria search for light (energy) sources using simple light intensity sensing.

Having enough time, our brains developed and we have about 5 know regions in the Visual Cortex, each responsible for a kind of feature (check on Mind Field)

Also, little is know about the learning process/otimization of a natural neuron but your question is on the data used...

Well, we cluster things in utility for survival: We detect human faces and perform person identification really well, this is one of the most advanced features of our visual cortex and this can be traced to our social needs which are intrinsically related to our survival ability. It is really important for us to identify the people that are friendly to us and those that may cause us harm.

When the object is brain diseases diagnosis using imaging, CNNs are already beating our brains.

So summarizing my answer: Fitness to environment allow us to define what to learn, correct predictions allow us to survive and evolve, while premature deaths avoid bad genes from propagating

Our environment provide us the label by Reinforced Learning + Genetic Algorithms.

Adding: We also developed the capability of propagating our knowledge (sometimes by genetic code and sometimes by teaching others).

Answer 3

I did not find a conclusive answer to your question. I present the closest content that I found, and my personal thoughts.

The closest I got was finding these well-cited papers:

1. 1997 How the brain learns to see objects and faces in an impoverished context

   Our results support psychological theories that perception is a conjoint function of current sensory input interacting with memory and possibly attentional processes.

2. 2004 The reverse hierarchy theory of visual perceptual learning,

   RHT proposes that naïve performance is based on responses at high-level cortical areas, where crude, categorical level representations of the environment are represented. Hence initial learning stages involve understanding global aspects of the task. Subsequent practice may yield better perceptual resolution as a consequence of accessing lower-level information via the feedback connections going from high to low levels (wiki page on Perceptual learning).

which lack the required comprehensiveness to answer the question. By going through the citations, I would say there is not yet a satisfying, and well-received answer to your question, which would usually lead to a highly-cited paper with a catchy title!

Among projects, I came across Project Prakash which seems interesting and related:

The goal of Project Prakash is to bring light into the lives of curably blind children and, in so doing, illuminate some of the most fundamental scientific questions about how the brain develops and learns to see (from here).

along with an interesting (but addressed as controversial) TED talk that shows how well blind adults that are cured recently manage to detect objects by putting emphasis on the role of motion (which objection detection methods based on single image lack). Here is an example of distinct objects that they detect, which is possibly worse than artificial neural networks.

Here are my thoughts regarding "the task" (which overlaps with @PedroHenriqueMonforte nicely put answer about evolution):

A "task" has an objective, a goal. What is the goal of brain at the highest level? To serve the gene for survival and reproduction. What if brain (eye, heart, etc.) fails at this task? The gene will be removed from the pool.

This is meta-learning, learning to learn. A pool of learners (genes that create brains that can learn to see) are constantly struggling to survive, where better (faster) learners have a higher chance of achieving the goal. This is the main supervision. At the extreme, the gene pool can get the job done by merely guessing the initial brain weights!

The most important take away here is that brains are evolving for about 450 million years. I think this alone suggests that not all of the visual understanding happens after birth. That is, animals are being born with good architectures and initial weights to begin with, analogous to being handed a network that is pre-trained on the task of survival and reproduction. From this perspective, visual training based on visual input would be more like a fine-tuning.