Uniformity of color and texture in an image

Question

I am new to the field of deep learning and have a problem in determining whether two images have uniform color and texture. For example, I have a

Master image -

Now, with respect to this image i need to determine whether the following images have uniform texture and color distributions -

image 1 -

image 2 -

image 3 -

I need to develop an algorithm which will evaluate these 3 images with the master image. The algorithm should approve the image 1 and reject image2 because of its color. And reject image 3 because of color and texture.

My approach for the problem was directly analyzing image for texture detection. I found that Local Binary Patterns method was good among all texture recognition methods (but I am not sure). I used its skimage implementation with opencv in python and found that the method worked.

from skimage import feature
import numpy as np
import cv2
import matplotlib.pyplot as plt

class LocalBinaryPatterns:
    def __init__(self, numPoints, radius):
        # store the number of points and radius
        self.numPoints = numPoints
        self.radius = radius

    def describe(self, image, eps=1e-7):
        # compute the Local Binary Pattern representation
        # of the image, and then use the LBP representation
        # to build the histogram of patterns
        lbp = feature.local_binary_pattern(image, self.numPoints,
            self.radius, method="uniform")
        (hist, _) = np.histogram(lbp.ravel(),
            bins=np.arange(0, self.numPoints + 3),
            range=(0, self.numPoints + 2))

        # normalize the histogram
        hist = hist.astype("float")
        hist /= (hist.sum() + eps)

        # return the histogram of Local Binary Patterns
        return hist


desc = LocalBinaryPatterns(24, 8)

image = cv2.imread("main.png")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
hist = desc.describe(gray)

plt.plot(hist,'b-')
plt.ylabel('Feature Vectors')
plt.show()

It detected the features and made a histogram of feature vectors. I plotted the histogram using matplotlib and clearly found that image 1 and image 2 texture features were almost similar to the master image. And image 3 texture features were not matching.

Then I started analyzing images for their color. I plotted the color histograms using opencv as -

import cv2
from matplotlib import pyplot as plt

def draw_image_histogram(image, channels, color='k'):
    hist = cv2.calcHist([image], channels, None, [256], [0, 256])
    plt.plot(hist, color=color)
    plt.xlim([0, 256])

def show_color_histogram(image):
    for i, col in enumerate(['b', 'g', 'r']):
        draw_image_histogram(image, [i], color=col)
    plt.show()

show_color_histogram(cv2.imread("test1.jpg"))

I found that color histogram of image 1 matched with master image. And color histograms of image 2 and 3 did not matched. In this way I figured out that image 1 was matching and image 2 and 3 were not.

But, I this is pretty simple approach and I have no idea about the false positives it will match. Moreover I don't know the approach for the problem is the best one.

I also want this to be done by a single and robust algorithm like CNN (but should not be computationally too expensive). But I have no experience with CNNs. So should I train a CNN with master images?. Please point me in the right direction. I also came across LBCNNs, can they solve the problem?. And what can be other better approaches.

Answer 1

this is pretty simple approach

Firstly, can you evaluate your scripts on more images to get an idea of how well it performs? If you get an acceptable classification accuracy (or e.g. a good F1 score), then there is no need to try out a CNN!

I have no idea about the false positives

Actually if you cannot evaluate your method like that, then a CNN is also not possible!

If you still want to try a deep learning method, bear in mind that you will generally need a lot of images, let's say at least 1000 in your "master image" training set - then hopefully a good percentage of images to test again (hold-out set / test set).

I don't know the approach for the problem is the best one

Your current method seems reasonable to me. There probably is "the best" method in general, so don't be too worried.

Somebody has written An Analysis of Deep Neural Networks for Texture classification - maybe that contains some ideas to get you started :-)

---

Here is a short video intro to one approch that introduced the Local Binary CNN, which was originally used for image classification, but perhaps could be adjusted to your problem.

Instead of aiming for target classes, you would simply need to map input to the labels, and could even relax the focus on colour/texture - the CNN would extract what it needs to learn the mapping.

NOTE: the downside of an end-to-end system like that is that you would no longer know what your model is using as its features to make its prediction! Colour? Texture? Raw RGB values in the top left corner?

This is actually where the LBCNN could shine, because its sparse (stochastically) binary kernels are much less likely to overfit to your training data, compared to a standard CNN.

While I couldn't find the code of the authors from the video above, here is an LBCNN implementation for face detection.