2
$\begingroup$

I would like to give some brief background for my question to avoid answers that explain the difference between fully connected nets and ConvNets.

I completed the first 3 courses in the deep learning specialization from deeplearning.ai (Coursera). Simultaneously I've worked through the first two home work assignments from Stanford's CS321n course for the Spring of 2017, as well as Michael Nielson's book on deep learning.

I believe I have a reasonable understanding of the architecture of both types of networks, and how to write them in python/tensoflow. I also implemented my own very simple fully connected feedfoward net in C where I was able to get 98% on MNIST.

I've also done some reading about this question in terms of which type of data is better suited for a ConvNet, and the answer I've come up with is similar to:

If the order of the input features matters to their meaning, then a ConvNet will work best, but if you replace the position of one column with another and the meaning is still intact, then a ConvNet will not work.

This means that spreadsheet data (if I've understood correctly) is not suited for a ConvNet.

Unfortunately I don't know any deep learning practitioners to ask this question: are there fully connected deep neural networks in the wild that are currently in use? The best example I can think of is financial data. If that's true, what are some others?

Improve this question
$\endgroup$
1
  • $\begingroup$ Are neural nets with dense layers in use? yes of course, that's one of the simplest possible layers, and found in almost any neural net architecture somewhere. CNNs also use them in additional to convolutional layers. What are you asking beyond that? $\endgroup$
    – Sean Owen
    Commented Jan 8, 2019 at 1:20

1 Answer 1

Reset to default
5
$\begingroup$

Yes, there are plenty use case where a fully connected network is more appropriate.

ConvNet works because the weights are shared in its kernels. It makes a lot intuitive sense to share the weights in image recognition task, because a flower at the top left of a picture has the same local pattern as the flower at the right bottom of the picture. What we want is to learn of pattern that is local.

However, this logic no longer applies when you consider program such as the famous iris classification problem. The idea of "local" simply doesn't exist for such data set. each of the features (i.e, petal width, petal length etc) are all "standalone" concept. In those case, it simply doesn't make sense to use ConvNet.

BTW, you can use ConvNet on financial data as well. As financial problem usually involves a time dimension, where the concepts of "local" does exist.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.