1
$\begingroup$

I want to identify subtle patterns in images using a convolutional neural net. I have seen several examples where people gave up reasoning that the pattern is not dominant or consistent enough to be picked up by a neural network. Since tuning these networks is very time consuming I wonder: What are methods to validate that the problem is solvable/the pattern can be recognised (or at least give an indication of the same)?

Background: My problem

My goal is to identify subtle differences in products based on their images. I want to break down a product image into a large variety of describing attributes. Now I need to find out which product attributes are well suited for identification.

For example, I want to distinguish collar types in clothing images. Therefore I need to figure out if the human understanding of collar types (waterfall, V-shaped, round etc.) are distinct and consistent enough to be properly identified by a neural net with human level performance.

Of course images always differ a little and the solvability is clearly related to the data available. But before collecting and cleaning hundreds of thousands of images I would like to find out if results will be any good.

Improve this question
$\endgroup$
4
  • $\begingroup$ "Very time consuming" is subjective on the equipment which is available to you, your level of expertise and whether you truly care about solving the problem. If you have access to a labeled dataset trying a few models to see if their is merit in pursuing the idea seriously should not take more than a few hours. $\endgroup$ – JahKnows Mar 21 '18 at 10:20
  • $\begingroup$ We can help you get started if you provide more details about your dataset and what you hope to detect from it. $\endgroup$ – JahKnows Mar 21 '18 at 10:21
  • 1
    $\begingroup$ take a look at here. $\endgroup$ – Media Mar 21 '18 at 13:56
  • $\begingroup$ @JahKnows thanks for your advice. I added more details about the problem at hand. $\endgroup$ – Gegenwind Mar 21 '18 at 16:15
1
$\begingroup$

Neural network will pick up on any patterns as long as your environment is not fully stochastic.

Here is what authors of Using Recurrent Neural Networks to Frecasting of Forex used to determine the amount of stochasticity:

page 7, bottom. "Hurst Exponent".

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for this solid reference. Can you tell how this can be applied to image data? $\endgroup$ – Gegenwind Mar 22 '18 at 10:09
1
$\begingroup$

It's much easier to identify projects where a neural network won't work than to identify projects where it will work. For example, if you have fewer than 10,000 records (highly subjective rule of thumb), then a neural net will probably overfit and not work. If a human struggles with the labels, then a neural network will probably struggle too.

In my subjective opinion, your problem sounds like it will be difficult for a neural network (or any model) to learn without a substantial amount of records. You would probably need several hundred thousand at a minimum, not including data augmentations like flipping images, etc, or something on the scale of an ImageNet dataset (i.e., millions). I say that because even as a human being I don't know what those collar types are that you described. On a dataset in the low tens of thousands you could probably train a convnet to identify shirts vs pants, but I highly doubt it could accurately detect collar types.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your answer, this was quite helpful (upvote). Did I get you right that solvability in your eyes only depends on data availability? In other words if I am able to increase the amount of data anything would be solvable this way? $\endgroup$ – Gegenwind Mar 21 '18 at 18:46
  • $\begingroup$ Yes, solvability can go up with substantially more data, but that's not a guarantee. Some problems are just inherently harder than others. For an obvious example, you could have millions of records and probably still not be able to "solve" stock market predictions. Having sufficient data is just a minimum requirement. $\endgroup$ – Ryan Zotti Mar 21 '18 at 19:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.