0
$\begingroup$

I am trying to make a convolutional neural network that classify images in two categories: with cats and without cats. It's the first time I am doing something like this and it seems I am having a problem.

What I was doing was to take some pictures, resize them to 100 by 200, apply a sobel filter to detect edges which worked, than make a pooling layer of size 2 by 2 which would reduce the complexity. I don't want high accuracy, for a beginner I think it's enough if this works for a few photos.

After applying the pooling layer, I created a fully connected layer. It had 2500 input neurons, 1000 hidden neurons in one layer and one output neurons. I used the sigmoid activation function. In the end I have to minimize this function in order to train my network: 2(y-sig(sig(X*W1)*W2))^2.

y is the expected result, X is the input vector, W1 and W2 are the two sets of weights and sig() is the sigmoid function. The design I thought of was that if the output was greater than 0.5 than there was a cat in the image otherwise not. The problem I am having is that after I generate the W1 and W2 in matlab using rand() all my values in the hidden layer are equal to 1 which is obvious as W1 always has positive values for some reason. After making some calculations in my head I realized that the output will always be equal to 1 because sig(X*W1)*W2 is a big number so, if I am correct, the network will identify a cat in almost every photo and training will not change much as the error is already small. My question is am I doing something wrong on my design or should I just find a way to have both negative and positive values for my weights? Thanks, and sorry for such a long text!

$\endgroup$
4
  • $\begingroup$ You got me confused a bit. You say that you want to make a CNN for image classification, but you don't have any convolutional layers in your model... $\endgroup$
    – Djib2011
    Apr 23, 2019 at 21:13
  • $\begingroup$ To my understanding a sobel filter (edge detection filter) can be viewed asa a convolution layer. It extracts fetaures from a photo, in this case edges. $\endgroup$
    – user6710
    Apr 24, 2019 at 9:34
  • $\begingroup$ Convolution layers have parameters that need to be trained; that's their whole point! Their goal is to extract the best features in order for an image to be classified. Additionally, multiple filters (not just one) need to be applied to an image. To answer your question, ideally want both your weights and X to be small numbers centered around 0 with unit variance so that you don't fall into the problem you mentioned. Hopefully, they will be kept small during training, but there are techniques that help enforce that. You also could change the 'sigmoid' for a 'ReLU' which does not saturate. $\endgroup$
    – Djib2011
    Apr 24, 2019 at 10:59
  • $\begingroup$ Thanks! Already used ReLu instead of sigmoid, and I chnaged the starting weights to -1 and the untrained network output is 0.5 which I think is great. I will look into convolution layers again, because it seems I didn't get it right. $\endgroup$
    – user6710
    Apr 24, 2019 at 11:30

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.