1
$\begingroup$

I use square loss to measure the difference between the output of the deconvolution network and the input image, use learning rate 0.01 and use SGD to optimize, code is written in tensorflow.

model_params = [["layer1",{"type":"conv_relu", "kernel":[3, 3, 3, 64],  "padding":"same"}],
                         ["layer2",{"type":"conv_relu", "kernel":[3, 3, 64, 64],  "padding":"same"}],
                         ["layer3",{"type":"max_pooling", "ksize":[1, 2, 2, 1],  "padding":"same"}], # ksize":[1, 2, 2, 1], 
                         ["layer4",{"type":"conv_relu", "kernel":[3, 3, 64, 128], "padding":"same"}],
                         ["layer5",{"type":"conv_relu", "kernel":[3, 3, 128, 128],  "padding":"same"}],
                         ["layer6",{"type":"max_pooling", "ksize":[1, 2, 2, 1], "padding":"same"}],
                         ["layer7",{"type":"conv_relu", "kernel":[3, 3, 128, 256],  "padding":"same"}],
                         ["layer8",{"type":"conv_relu", "kernel":[3, 3, 256, 256],  "padding":"same"}],
                         ["layer9",{"type":"conv_relu", "kernel":[3, 3, 256, 256], "padding":"same"}],
                         ["layer10",{"type":"conv_relu", "kernel":[3, 3, 256, 256],  "padding":"same"}],
                         ["layer11",{"type":"max_pooling", "ksize":[1, 2, 2, 1], "padding":"same"}],
                         ["layer12",{"type":"conv_relu", "kernel":[3, 3, 256, 1024],  "padding":"same"}],
                         ["layer13",{"type":"conv_relu", "kernel":[1, 1, 1024, 1024], "padding":"same"}],
                         ["layer14",{"type":"deconv_relu", "kernel":[3, 3, 256, 1024],  "padding":"same"}],
                         ["layer15",{"type":"demax_pooling"}],
                         ["layer16",{"type":"deconv_relu", "kernel":[3, 3, 256, 256],  "padding":"same"}],
                         ["layer17",{"type":"deconv_relu", "kernel":[3, 3, 256, 256],  "padding":"same"}],
                         ["layer18",{"type":"deconv_relu", "kernel":[3, 3, 256, 256],  "padding":"same"}],
                         ["layer19",{"type":"deconv_relu", "kernel":[3, 3, 128, 256],  "padding":"same"}],
                         ["layer20",{"type":"demax_pooling"}],
                         ["layer21",{"type":"deconv_relu", "kernel":[3, 3, 128, 128],  "padding":"same"}],
                         ["layer22",{"type":"deconv_relu", "kernel":[3, 3, 64, 128],  "padding":"same"}],
                         ["layer23",{"type":"demax_pooling"}],
                         ["layer24",{"type":"deconv_relu", "kernel":[3, 3, 64, 64],  "padding":"same"}],
                         ["layer25",{"type":"deconv_relu", "kernel":[3, 3, 3, 64],  "padding":"same"}],
                         ]  

but the loss did not go down,

2016-03-20 16:36:19.764441: step 0, loss = 54437.99 (24.1 examples/sec; 5.306 sec/batch)
2016-03-20 16:36:42.029972: step 10, loss = 54460.74 (65.3 examples/sec; 1.959 sec/batch)
2016-03-20 16:37:02.488069: step 20, loss = 54398.99 (61.1 examples/sec; 2.096 sec/batch)
2016-03-20 16:37:25.320226: step 30, loss = 54171.65 (61.0 examples/sec; 2.100 sec/batch)
2016-03-20 16:37:45.594397: step 40, loss = 54180.36 (62.6 examples/sec; 2.044 sec/batch)
2016-03-20 16:38:08.296269: step 50, loss = 54444.09 (63.4 examples/sec; 2.020 sec/batch)
2016-03-20 16:38:28.625820: step 60, loss = 54644.72 (61.0 examples/sec; 2.099 sec/batch)
2016-03-20 16:38:50.959914: step 70, loss = 54447.85 (61.6 examples/sec; 2.077 sec/batch)
2016-03-20 16:39:11.174660: step 80, loss = 54334.46 (61.7 examples/sec; 2.074 sec/batch)
2016-03-20 16:41:16.831728: step 140, loss = 54293.51 (67.0 examples/sec; 1.911 sec/batch)
2016-03-20 16:41:38.710691: step 150, loss = 54386.80 (59.9 examples/sec; 2.138 sec/batch)

How to check where go wrong or maybe someone have seen this problem before? enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.