I am trying to replicate the general image classification model used in a paper that I cite later below. The following image is an extract from a paper that proposes a novel method of performing image classification. I do not have the technical capacity to recreate what they have done, so i intend to build something simpler in keras but with similar structure, for a course project on deep learning.
The author uses 4 parallel N.N models, fed by the same input image, but pre-processed in different ways. It is not shown, but the pre-processed images are passed through a "window convolution" before being passed into the first layer, the k-means encoder, hence the 192 channels at the input.
Notice, from the bottom of the image the author highlights the use of supervised and unsupervised training. In the unsupervised training portion there are "tiled-CNN" and "Recursive-tiled-CNN" layers.
I am familiar with building a CNN models for classifiers in keras, but that involved supervised training by labeled samples. My first question is how can the kernels of the CNN in this particular model (TCNN and R-TCNN) be trained without labelled data?
In my project I intend to replace the R-TCNN and TCNN layers with simpler CNN layers. Still then, i'm not sure how to train the CNN in an unsupervised setting.
For the moment I am assuming that I am mistaken and that the CNNs are intended to be trained with labelled data. I am also assuming that I need to train the layers individually in a sequential manner starting from the lowest layer and then then merge the models. Does this seem like a reasonable approach? I outline it below..
1)Train the 192 kernels of window convolution in a supervised manner, with labeled data, in a classifier setting to determine best kernels.
2)Train the k-means clustering algorithm with the image data that has been processed by the prior window convolution. The image labels would be ignored.
3)Train the auto-encoder with the results of the K-means clustering. Image labels again would be ignored.
4)Train the CNN in a supervised manner with the labelled image data after it has passed though all prior layers.
Paper: V. D. Nguyen, H. Van Nguyen, D. T. Tran, S. J. Lee and J. W. Jeon, "Learning Framework for Robust Obstacle Detection, Recognition, and Tracking," in IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 6, pp. 1633-1646, June 2017.