0
$\begingroup$

Below is the SS of the custom function I am trying to apply on every image of the batch and the custom Layer

def geo_features( input_img ):
    print( "INPUT IMAGE SHAPE:", input_img.shape, type( input_img ) )
    detector = dlib.get_frontal_face_detector()
    predictor = dlib.shape_predictor( config.SHAPE_PREDICTOR_DAT_PATH )
    cv2.imshow( 'test', input_img )
    cv2.waitKey( delay = 0 )
    img = cv2.resize(
        input_img,
       (
         config.TARGET_SIZE['CV2_LANDMARKS_RESIZE'][0],
         config.TARGET_SIZE['CV2_LANDMARKS_RESIZE'][1]
       )
    )
    row = [ ( -100, -100 ) ]
    faces = detector( img )
    .
    .
    .


class GeoFeatures( keras.layers.Layer ):
    def __init__( self ):
        super( GeoFeatures, self ).__init__()
        print( "BUILDING CUSTOM LAYERS")

    def call( self, inputs ):
        print( "INSIDE CALL METHOD" )
        return tf.map_fn( fn = tf.py_function( geo_features ), elems = inputs)

    def compute_output_shape(self, input_shape):
        return ( input_shape[0], config.NUM_GEOMETRIC_FEATURES )

The issue I am facing is, the GeoFeatures call method is being called before compilation as evident from the below pics:- enter image description here

Below is part of the training code. (NOTE:- model_dispatcher.return_mode() is just returning the keras.Model and nothing else).

train_data = train_datagen.flow_from_dataframe(
            dataframe = train_df,
            directory = None,
            x_col = "Image",
            y_col = "Label",
            target_size=(config.TARGET_SIZE[config.MODELS[num_fold-1]][0], config.TARGET_SIZE[config.MODELS[num_fold-1]][1]),
            class_mode = "categorical",
            shuffle = True,
            color_mode = config.COLOR_MODE[config.MODELS[num_fold-1]],
            batch_size = config.BATCH_SIZE,
            seed = 42
        )

        val_data = val_datagen.flow_from_dataframe(
            dataframe = val_df,
            directory = None,
            x_col = "Image",
            y_col = "Label",
            target_size=(config.TARGET_SIZE[config.MODELS[num_fold-1]][0], config.TARGET_SIZE[config.MODELS[num_fold-1]][1]),
            class_mode = "categorical",
            color_mode = config.COLOR_MODE[config.MODELS[num_fold-1]],
            shuffle = True,
            batch_size = config.BATCH_SIZE,
            seed = 42
        )

        print( train_data.class_indices )
        print( "DISPATCHING MODEL" )
        model = model_dispatcher.return_model( num_fold )
        mc, reduce_lr = return_callbacks(
            num_fold,
            isBest
        )
        print( "NOT YET COMPILED")
        opt = return_opt( 'adam', learning_rate = 1e-3 )
        model.compile(
            optimizer = opt,
            loss = 'categorical_crossentropy',
            metrics = ['accuracy']
        )
        model.fit( .. )
        .
        . 
        .
        .

As can be seen clearly, the compilation is not yet done still the call method of layer is being called resulting in an error.

For more clarity below is the model which is incorporating the custom layer. Full model code is not shown as it is irrelevant as the custom layer is used only once here. At the last, the customXceptionWithGeo returns Keras.Model( inputs, outputs )

def customXceptionWithGeo( TARGET_SIZE ):
    inputs = tf.keras.Input(TARGET_SIZE)

    y = GeoFeatures()( inputs )
    y = Dense( 16, activation = 'relu' )( y )
    y = Dense( 32, activation = 'relu' )( y )
    y = Dropout( 0.4 )( y )
    y = Dense( 64, activation = 'relu' )( y )
    y = Dense( 32, activation = 'relu' )( y )
    y = Dense( config.NUM_CLASSES )( y )
    .
    .
    x = ...
    .
    .
    x = layers.add( [ x, y ] )
    outputs = Activation('softmax', name='predictions')(x)
    return keras.Model( inputs, outputs )
$\endgroup$
2
  • $\begingroup$ Instead of implementing all the logic of geo_features() method in a custom Keras Layer, I'd suggest you to create another Input layer for your model and pass the data to this input layer ( which is already processed by calling geo_features() method ). $\endgroup$ – Shubham Panchal May 11 at 13:44
  • $\begingroup$ @ShubhamPanchal I understand that that can be done. But, as you can see in customXceptionWithGeo model that, this geometric model's output is getting merged into another model( defined from x = ... ). Let's call this another model as "Main model" The take is, I am applying feature extraction on the augmented image( which is getting fed into the Main model ) for the geometric model and the same augmented image is getting fed into the main model too. At last, the outputs of both models are merged. $\endgroup$ – Mrityu May 11 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.