8
$\begingroup$

I want to extract parameters from an image using a neural network.

Example:

Given an image of a brick wall the NN should extract the width and height of the bricks, the color and the roughness.

I can generate images for given parameters to train the NN and want to use it to extract the parameters from an actual image.

I've looked into CNNs. Can I perform this task with them? Do I need special learning algorithms to extract multiple parameters instead of classification? Are there any NNs that are designed for such tasks?

$\endgroup$
2
  • $\begingroup$ Could you attach a few sample images you've got in mind? CNNs might be overkill for the task, but then on the other hand worth studying and experimenting with if your top priority is the usage of neural networks in contrast to "traditional" CV algorithms. $\endgroup$ – NikoNyrh Jun 13 '16 at 14:13
  • $\begingroup$ @NikoNyrh could provide sample images, but I want to use the technique for various classes of textures. The goal is to extract shader parameters from images. The brick example contains mostly intuitive parameters, but other shaders will probably use parameters that can't be easily obtained by a well engineered algorithm. I'm currently testing different approaches and wanted to give neural networks a try as I can generate "infinite" training data. $\endgroup$ – H4kor Jun 13 '16 at 17:05
8
$\begingroup$

A CNN could be a good choice for this task if you expect variation in the original image scale, rotation lighting etc, and also have a lot of training data.

The usual CNN architecture is to have convolutional layers close to the input, and fully-connected layers in the output. Those fully-connected layers can have the output arranged for different classification or regression tasks as you see fit. Predicting the values of parameters describing the image is a regression task.

If you want accurate measures of size, you may need to avoid using max pooling layers. Unfortunately, not using pooling will make your network larger and harder to train - you might get away with strided convolution instead if that is a problem for you.

If your input images are very simple and clear (because they are always computer generated), then other approaches may be more reliable. You may be able to reverse-engineer image production and derive simple rules such as identifying lines, corners, circles and other easy-to-filter image components, and make direct measurements. There may also be a middle ground in complexity where extracting this data as features and using it to train a simple NN (or other ML model) will have good performance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.