0
$\begingroup$

I am trying to build a model to predict how much time does it cost to produce a component. I am using 600 images for training and validation. I also use data augmentation. I tried many combinations but mean absolute percentage error does not become better than 87. Here is the model building code below, any suggestion is highly appreciated.

def preprocess_image(img, file):
    img = cv2.resize(img, (224, 224))
    img = change_black_bg_to_white(img, file)
    img = clahe(img)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    th, threshed = cv2.threshold(gray, 240, 255, cv2.THRESH_BINARY_INV)

    ## (2) Morph-op to remove noise
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (11,11))
    morphed = cv2.morphologyEx(threshed, cv2.MORPH_CLOSE, kernel)

    ## (3) Find the max-area contour
    cnts = cv2.findContours(morphed, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)[-2]
    cnt = sorted(cnts, key=cv2.contourArea)[-1]

    ## (4) Crop and save it
    x,y,w,h = cv2.boundingRect(cnt)
    dst = img[y:y+h, x:x+w]
    sharpen_kernel = np.array([[-1,-1,-1], [-1,9,-1], [-1,-1,-1]])
    sharpen = cv2.filter2D(dst, -1, sharpen_kernel)
    dst = cv2.cvtColor(sharpen, cv2.COLOR_BGR2GRAY)
    return dst

def clahe(img):
    lab= cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
    l_channel, a, b = cv2.split(lab)

    # Applying CLAHE to L-channel
    # feel free to try different values for the limit and grid size:
    clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
    cl = clahe.apply(l_channel)

    # merge the CLAHE enhanced L-channel with the a and b channel
    limg = cv2.merge((cl,a,b))

    # Converting image from LAB Color model to BGR color spcae
    enhanced_img = cv2.cvtColor(limg, cv2.COLOR_LAB2BGR)

    # Stacking the original image with the enhanced image
    result = np.hstack((img, enhanced_img))
    return enhanced_img

 def change_black_bg_to_white(img, file): 
    img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB) 
    if (file == 'MS20001P4-636-WH.jpg') or (file == 'MS20001P4-2772-WH.jpg'): #turuncu
        lower_white = np.array([160,  70, 0], dtype=np.uint8)
        upper_white = np.array([215,  115, 55], dtype=np.uint8)

    elif (file == 'MS20001P4-1480-WH.jpg') or file == 'MS20001P4-472-WH.jpg': #mavi
        lower_white = np.array([0, 70, 70], dtype=np.uint8)
        upper_white = np.array([255, 205, 205], dtype=np.uint8)

    elif file == 'MS20001P4-1850-WH.jpg': #pembe
        lower_white = np.array([100, 0, 150], dtype=np.uint8)
        upper_white = np.array([255, 120, 255], dtype=np.uint8)
        
    elif file == 'EU5701641024100_Sketch.jpg': #gri
        lower_white = np.array([170, 170, 170], dtype=np.uint8)
        upper_white = np.array([210, 210, 210], dtype=np.uint8)

    else:
        lower_white = np.array([47,  47, 99], dtype=np.uint8)
        upper_white = np.array([56, 56, 106], dtype=np.uint8)
        
    mask = cv2.inRange(img, lower_white, upper_white)  # could also use threshold
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3)))  # "erase" the small white points in the resulting mask
    mask = cv2.bitwise_not(mask)  # invert mask

    # load background (could be an image too)
    bk = np.full(img.shape, 255, dtype=np.uint8)  # white bk

    # get masked foreground
    fg_masked = cv2.bitwise_and(img, img, mask=mask)

    # get masked background, mask must be inverted 
    mask = cv2.bitwise_not(mask)
    bk_masked = cv2.bitwise_and(bk, bk, mask=mask)

    # combine masked foreground and masked background 
    final = cv2.bitwise_or(fg_masked, bk_masked)
    mask = cv2.bitwise_not(mask)  # revert mask to original
    return final

train_datagen = ImageDataGenerator(rescale=1./255,
                                    shear_range=0.2,
                                    zoom_range=0.2,
                                    rotation_range=30,
                                    #horizontal_flip=True,
                                    #vertical_flip=True,
                                    validation_split=0.2
                                    )

test_datagen = ImageDataGenerator(rescale=1./255)

train_set = train_datagen.flow_from_dataframe(train_df,
                                                directory=train_path,
                                                x_col='DOSYA',
                                                y_col='TIME',
                                                target_size=(224, 224),
                                                batch_size=4,
                                                class_mode='raw',
                                                subset='training',
                                                drop_duplicates = False)
                                                
validation_set = train_datagen.flow_from_dataframe(train_df,
                                                directory=train_path,
                                                x_col='DOSYA',
                                                y_col='TIME',
                                                target_size=(224, 224),
                                                batch_size=4,
                                                class_mode='raw',
                                                subset='validation',
                                                drop_duplicates = False)

test_set = test_datagen.flow_from_dataframe(test_df,
                                                directory=test_path,
                                                x_col='DOSYA',
                                                y_col='TIME',
                                                target_size=(224, 224),
                                                batch_size=1, class_mode = 'raw',
                                                drop_duplicates = False)

model = Sequential()

def convolutional_layers(model):
    model.add(Conv2D(64, (3, 3), padding='same', input_shape=(224, 224, 3), activation='relu')) 
    model.add(BatchNormalization())

    model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
    model.add(BatchNormalization())

    model.add(Conv2D(32, (3, 3), padding='same', activation='relu'))
    model.add(BatchNormalization())
    model.add(MaxPool2D(pool_size=(2, 2)))

    model.add(Conv2D(16, (3, 3), padding='same', activation='relu'))
    model.add(BatchNormalization())

    model.add(Conv2D(16, (3, 3), padding='same', activation='relu'))
    model.add(BatchNormalization())
    model.add(MaxPool2D(pool_size=(2, 2)))
    return model


def neural_layers(model):
    model.add(Flatten())
    model.add(Dense(units = 128, activation='relu'))

    model.add(Dense(units = 64, activation='relu'))
    model.add(Dropout(0.25))

    model.add(Dense(units = 16, activation='relu'))
    model.add(Dropout(0.25))

    model.add(Dense(units = 1, activation='linear'))
    return model


model = convolutional_layers(model)
model = neural_layers(model)

model.compile(
    loss=MeanSquaredError(),
    optimizer=Adam(lr=0.0001),
    metrics=[MeanSquaredError(), MeanAbsolutePercentageError()]
    )

batch_size = 4
history = model.fit_generator(train_set, steps_per_epoch=(train_set.samples//batch_size), epochs= 100, 
                             validation_data = validation_set, 
                             callbacks=[es,mc, rlr], workers=16)
Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ What sort of error would you expect your model to achieve giving the data you have, e.g. what would the human error be? In addition, only having 600 images to train an neural network on will probably be difficult. $\endgroup$
    – Oxbowerce
    Oct 10, 2022 at 13:58
  • $\begingroup$ Since it is a percentage error, I think it means my error rate is 80%. So it is an awful score. Do you see any problems in the archtiecture I built $\endgroup$
    – misafirmisim
    Oct 10, 2022 at 13:59
  • $\begingroup$ Also I should state, I am using images of the components that will be produced by workers. I am trying to build a model to predict how much time does it take to create each component. $\endgroup$
    – misafirmisim
    Oct 10, 2022 at 14:03
  • 1
    $\begingroup$ It might simply be that given the data a model cannot do any better because the problem is very difficult. That's why I asked how well a human would do when predicting given the data since this could then be used as a benchmark to compare the model's error with. $\endgroup$
    – Oxbowerce
    Oct 10, 2022 at 14:37
  • $\begingroup$ I will ask the business partners about expected human performance. Do you think it would be naive to expect model generalize better than a human? $\endgroup$
    – misafirmisim
    Oct 11, 2022 at 6:29

2 Answers 2

Reset to default
2
$\begingroup$

If you have tried a lot of different architectures and augmentation and you still can't improve your accuracy, it seems like you reached the limits of your model.

What an object looks like indeed does correlate with how much time it might need to make, but there are definitely other factors at play. Some complex-looking objects might be easier to make and the inverse is also true.

You can try adding more data (maybe raw material cost or something else domain-specific) after the convolutional layers are done learning what they can from the images.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I intend to add some excel data about length, width, material of component. But when I only use excel features and build machine learning models(like xgboost and random forest) mape error rate is around 80. So, do you think concatenating two bad models can led to a good model? Or I am wasting my time? $\endgroup$
    – misafirmisim
    Oct 10, 2022 at 14:29
  • 1
    $\begingroup$ Two "bad" models can lead to a better one all the time (depending of course on how bad they actually are). Boosting regressors and Random Forests use this principle in some form for their predictions. You can try stacking multiple models (for example use Random Forest to predict the numeric data and then use the neural networks to make another set of predictions, then use these predictions as input to train another model. $\endgroup$
    – liakoyras
    Oct 10, 2022 at 14:34
2
$\begingroup$

Observations -

  • Last layer's activation should be Linear
  • Use MSE as a Loss
  • LR is too low
  • Also, one of the Conv layers is using linear activation function, make it relu.
Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.