I'm working on a very small scale pet project in which inputs are essentially sets of (x, y) pairs, and are to be classified into categories, using deep learning, specifically using Keras (I know this may not be the best for this, but it's more of a proof of concept / I want to try it out).

However, I'm not sure how to go about representing the data.

I'm starting with a simple classification problem (i.e. if (a, b) is a feature of a sample, and a and b are both within 5% of a certain (c, d), then they are a positive example, and not otherwise), but I'm not sure how to represent the data such that the network can learn this.

I was thinking of doing one-hot encoding, but then the dimensionality of the data may grow immensely (x and y both take on values in a continuous interval), and I'm worried that it would not generalize well to data augmentation (I would augment the data by adding noise to each x, y).

Any ideas?

  • 1
    $\begingroup$ This is an interesting problem! A little more information would help. The inputs are sets of pairs. How large are the sets? Are they around the same size or do they vary wildly? Importantly, does the order of the set matter at all? This is a very different problem if the order matters than if it doesn't. $\endgroup$
    – Matthew
    Oct 26 '18 at 18:26
  • $\begingroup$ Is it possible to give a more explicit example of “if (a, b) is a feature of a sample, and a and b are both within 5% of a certain (c, d), then they are a positive example”? I’m not sure what you mean by both a and b (both real numbered elements of a tuple?) being within 5% of (b, c), as I don’t understand what is meant by a real number being within 5% of a tuple. $\endgroup$ Aug 22 '20 at 22:04

Your features are already numerical, so you can simply pass your array of numbers into a MLP. For instance, if you have 5 sets of (x,y) coordinates per sample, your MLP will have 10 inputs.

One-hot encoding is used to convert categorical features into numerical values. This image from Kaggle does a great job of illustrating how you might use one-hot encoding:

one-hot encoding

If you were to directly convert categorical values to numerical values, that would imply that one color is greater than another, so we need to use a boolean array instead.

As for the variable length feature vector, it is difficult to give an exact answer without knowing more details about your problem, but here are general possible solutions:

  • use a recurrent neural network
  • use distance-based regression/classification algorithms such as k nearest neighbors
  • transform your data into feature vectors of fixed length (which may not be reasonable depending on your data and what it represents)

A quick high level introduction to neural networks might be helpful.

  • $\begingroup$ Thanks for the response! However, my data is also of variable length (each sample could have as many x/y pairs as necessary. I'm also not sure how well it'd fare with the dependence of y on x for each value). I'm more curious about how to represent the facts that the data can have an variable amount of samples, where the order doesn't matter $\endgroup$
    – Alex
    Apr 27 '18 at 14:19
  • $\begingroup$ I updated the answer to address this. I could only provide very general solutions to this problem, but I might be able to offer better help if you describe the specific problem you are trying to solve and what your feature vectors represent. $\endgroup$ Apr 28 '18 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.