12
$\begingroup$

So far there are many interesting applications for deep learning in computer vision or natural language processing.

How is it in other more traditional fields? For example, I have traditional socio-demographic variables plus maybe a lot of lab measurements and want to predict a certain disease. Would this be a deep learning application if I have lots of observations? How would I construct a network here, I think all the fancy layers (convolutional etc.) are not really necessary?! Just make it deep?

On my specific data set, I tried some common machine learning algorithms like random forests, gbm etc with mixed results regarding accuracy. I have limited deep learning experience with image recognition.

$\endgroup$
  • 1
    $\begingroup$ You may be better off looking at a different approach, e.g. XGBoost, depending on how much is "lots of observations". Can you clarify whether your goal is to specifically to try deep learning approaches, or to get best accuracy? $\endgroup$ – Neil Slater Mar 8 '17 at 12:00
  • $\begingroup$ @NeilSlater my goal would be to achieve higher accuracy than established methods like xgboost, if that is possible in such a case $\endgroup$ – spore234 Mar 8 '17 at 13:00
  • 1
    $\begingroup$ It is possible, but in my experience not likely unless you really do have a lot of data. $\endgroup$ – Neil Slater Mar 8 '17 at 13:01
  • $\begingroup$ Matlab provides documentation about "Deep Learning Tips and Tricks". I had the same question and the page provided very useful guide along good examples. for example you may need sequence to sequence / time series classification/regression using deep learning. $\endgroup$ – Sami Navesi Sep 25 '18 at 18:33
10
$\begingroup$

Yes you can use deep learning techniques to process non-image data. However, other model classes are still very competitive with neural networks outside of signal-processing and related tasks.

To use deep learning approaches on non-signal/non-sequence data, typically you use a simple feed-forward multi-layer network. No need for convolutional layers or pooling layers. The best architecture other than that needs to be explored with cross-validation, and can be time-consuming to discover as deep NNs take a lot of computation to train.

In my experience attempting to use deep(-ish, typically ~ 5 layers) neural networks in Kaggle competitions:

  • Dropout is still highly effective for regularisation and improving accuracy

  • Input normalisation - usually to mean 0, standard deviaton 1, is important

  • Hidden layer activation functions can make a difference. Although ReLU reduces some problems with vanishing gradients, in my experience it is less robust with non-signal data and you will want some other form. If you have only a few layers, then sigmoid or tanh still work OK. Otherwise, look into leaky ReLU, PReLU, ELU and other ReLU variants that attempt to patch its problems with "dead" neurons.

  • Make use of optimisers designed for deep learning, such as Adam, Adagrad or RMSProp

  • Use a weight initialisation approach that works with deep learning, such as Glorot.

  • Consider using Batch Normalisation layers. Not something I have much experience with, but I have seen other people do well with this approach.

Despite all this, XGBoost can routinely and easily beat deep NNs with minimal tuning and training effort in comparison (depending of course on the problem and the data you have). If accuracy is everything to you though, it is possible - although not guaranteed - that an ensemble of deep NNs and other model such as XGBoost will perform better than either singly.

| improve this answer | |
$\endgroup$
1
$\begingroup$

A network can be appropriate for classification purposes. For this, you need to be able to define a training-set and a test-set of your data that represents the data the network will be asked to classify in production. This determines if you can get a bad, a reasonable or a good working network.

I consider terms as "deep-learning" as misleading: a network does not learn, you can only train it.

Assuming you can create a training and test set, on high level you can use a

  • Multi-layer: if your data has no order and structures have a fixed position.

  • Recursive networks: if the order of the data is of importance for the classification

  • Convolution: if your data has structures like in images but there is no fixed position.

Getting a good setup, like the number of layers, requires trial and error; it is a kind of black magic.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.