1
$\begingroup$

In lecture, we talked about “parameter sharing” as a benefit of using convolutional networks. Which of the following statements about parameter sharing in ConvNets are true? (Check all that apply.)

  1. It allows parameters learned for one task to be shared even for a different task (transfer learning).
  2. It reduces the total number of parameters, thus reducing overfitting.
  3. It allows gradient descent to set many of the parameters to zero, thus making the connections sparse.
  4. It allows a feature detector to be used in multiple locations throughout the whole input image/input volume.

Here are the correct answers:

  1. It reduces the total number of parameters, thus reducing overfitting.
  2. It allows a feature detector to be used in multiple locations throughout the whole input image/input volume.

Why isn't the following answer also correct:

  1. It allows parameters learned for one task to be shared even for a different task (transfer learning). Doesn't ConvNets allow parameters to be shared, detecting the similar features of different images?
$\endgroup$
  • $\begingroup$ Hi Oscar, your question requires some clarification: The numbers listed under "correct answers" do not match the numbers given in your list of potential answers. Moreover, you are asking why the answer regarding the number of parameters is not correct but at the same time you are listing it under correct answers. Can you edit your question to remove these issues? $\endgroup$ – Sammy Sep 2 '20 at 14:26
  • 1
    $\begingroup$ @Sammy sorry for the confusion! I have fixed the question. $\endgroup$ – Z Oscar Sep 4 '20 at 8:56
0
$\begingroup$

The point is that parameters sharing is not the same as reusing some learnt weights in a task for another task (a.k.a transfer learning); it is, transfer learning can be used with convolutional neural networks, but parameters sharing does not mean transferring knowledge from one task to another. For a detailed definition of parameters sharing, look at this documentation from Stanford university , section Parameter Sharing

As explained in that page, you can think of it as a tool for reducing the number of weights while learning in a specific task (so forget about traqnsferring these weights to another task, which could be the case, but not the meaning of this concept).

An intuitive way of reasoning the concept could be: parameter sharing assumption is relatively reasonable: If detecting a horizontal edge is important at some location in the image, it should intuitively be useful at some other location as well due to the translationally-invariant structure of images

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.