3
$\begingroup$

I have a rectangular numeric dataset, and I'm applying a multilayer perceptron to it. I'm having success, but I'm now looking to see what other architectures I can apply.

Much of deep learning seems applied to loosely-structured data -- sequences, text, images -- and everyone is having a lot of fun working with a variety of interesting models...at least when they have a problem that fits these models.

What about basic, row/column datasets. What are some of the canonical models to be used with this kind of data, apart from tweaking the layers of a basic MLP?

$\endgroup$
  • $\begingroup$ Images are strongly structured. Neural networks can deal with structured prediction just fine. What kind structure does you data have? $\endgroup$ – Emre Aug 10 '17 at 16:37
  • $\begingroup$ 35 features...3 class output. Very basic...all data is numeric $\endgroup$ – Monica Heddneck Aug 10 '17 at 16:39
  • $\begingroup$ You don't make it sound very structured, so why can't you use an MLP? I assume you know that each "row" (datum) is fed separately? $\endgroup$ – Emre Aug 10 '17 at 16:41
  • $\begingroup$ I can use an MLP...what else can I also use? My options feel limited. $\endgroup$ – Monica Heddneck Aug 10 '17 at 16:42
  • 1
    $\begingroup$ @MonicaHeddneck actually based on your debates I suggest using linear svm. linear svm is best for data which are linearly separable. whenever I try to fit a model I try linear models to see whether the data in the input dimension can be separated linearly (using liner models) or not. This may help you $\endgroup$ – Media Aug 10 '17 at 17:12
3
$\begingroup$

Fancy deep learning architectures mostly work by exploiting structure in your data. Permutation invariance or equivariance, temporal structure and spatial structure come to mind. Maybe some features are based on a set of the same objects? Most of the benefits come from sharing weights and learning a shared representation. Another potential benefit is the flexibility in output, although that is less relevant in your case. If you have a lot more labeled data on another task with the same features, you could pretrain a larger model on that task and then fine tune that network on your current task. For the rest it is difficult to say without showing some examples.

$\endgroup$
1
$\begingroup$

Given 35 numeric features and 3 classes, perhaps first try a variant of SVM or random forest to get a handle on the data - neither of these are deep learning, but they are fast and good for benchmarking/troubleshooting:

If you've already done that, you could use a restricted boltzmann machine to build a generative model of the data, or an unsupervised algorithm like an autoencoder or self-organising map to reduce the dimensionality of the data, then use that as the input to your MLP/SVM/softmax classifier.

There are other options out there but it might be worth finding out what sort of results you get on the faster and more explanatory algorithms before building a full CNN etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.