0
$\begingroup$

I'm looking for an approach to classify a similar dataset to the exposed next. Let's say we have an image with some elements inside it (imagine a large building footprint with several structures). What I want to classify are those individuals but taking into account the others. The main issue here is that some individuals are common between them, but having the whole context any can give some insight. Each element is very simple, just some colored lines over a black background, so if the footprint has n elements I extract n 256x256 images to be classified.

The approach tested so far is the individual classification of each image with a classic CNN (namely CNN1). Then, in a next step I take each feature map of the last fully connected layer of CNN1 and compose them in order to create the individual context of the target individual, so the output would be stack(feature_map_target, max_pool(feature_map_1, feature_map_2,..., feature_map_n-1)), with feature_map_i the footprint indviduals discarding the targeted one. Lastly, this context is the input of a new CNN2 with a couple of fully connected layers which performs a classification at its end.

For now I'm not succeeding (around 65% of accuracy), because there are some individuals very similar between them and I'm loosing the context trained in CNN2 when classifying. Any light here?

Thanks!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

You can perform individual image classification using as input the whole set of related images as well. In order to do this, your initial input will be some 3d array of stacked 256x256 matrices and the first layer could be either a 3d convolution or multiple 2d convolutions with some combination function at some point of the structure.

Another way would be having a first network structure that processes each image and outputs a dense vector of dimension D and stacking (or averaging) those vectors and using this as inputs to a sequential structure. This way you would be training the process as a whole instead of performing two separated tasks.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the response. The second approach is actually what I'm using. That stacking you referred is what I'm max pooling. I guess both networks should be kind of connected so I can take advantage of the second output to backpropagate it to the first one, right? $\endgroup$
    – Alejandro Graciano
    Nov 7 at 13:21
  • $\begingroup$ Yes, you don't need to train the first structure to generate some classification, just backpropagate on the final output of the combined structure. $\endgroup$
    – Giovanni Amorim
    Nov 7 at 13:55
  • $\begingroup$ The max pooling you are performing can make the model ignore some information that you would want it to take into consideration. Consider changing it to some dense layer of some other form of aggregation, like averaging. $\endgroup$
    – Giovanni Amorim
    Nov 7 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.