I am trying to make a convolutional neural network that classify images in two categories: with cats and without cats. It's the first time I am doing something like this and it seems I am having a problem.
What I was doing was to take some pictures, resize them to 100 by 200, apply a sobel filter to detect edges which worked, than make a pooling layer of size 2 by 2 which would reduce the complexity. I don't want high accuracy, for a beginner I think it's enough if this works for a few photos.
After applying the pooling layer, I created a fully connected layer. It had 2500 input neurons, 1000 hidden neurons in one layer and one output neurons. I used the sigmoid activation function. In the end I have to minimize this function in order to train my network: 2(y-sig(sig(X*W1)*W2))^2.
y is the expected result, X is the input vector, W1 and W2 are the two sets of weights and sig() is the sigmoid function. The design I thought of was that if the output was greater than 0.5 than there was a cat in the image otherwise not. The problem I am having is that after I generate the W1 and W2 in matlab using rand() all my values in the hidden layer are equal to 1 which is obvious as W1 always has positive values for some reason. After making some calculations in my head I realized that the output will always be equal to 1 because sig(X*W1)*W2 is a big number so, if I am correct, the network will identify a cat in almost every photo and training will not change much as the error is already small. My question is am I doing something wrong on my design or should I just find a way to have both negative and positive values for my weights? Thanks, and sorry for such a long text!