I've tried a few ways to do my multi-class classification. For categorical data, I used the embedding technique with Tensorflow, which moves the entity closer with its similarity. This technique provides me with approximately 25%-30%, which is low.

For the numerical data, I used the KNN algorithm that gave me roughly 40% accuracy. I am wondering is there any way to "combine" these two techniques together to achieve a better result. For example, perhaps using the probability given by the KNN algorithm to form a layer concatenated with the embedding layer. Then, use the Dense layer to further train these data.

I've searched on the Internet. It's not the technique of ensembling, which averages the accuracy of each model. It's more like concatenating the layer together.

Any help is highly appreciated.

enter image description here


1 Answer 1


If I understand well, you label encode categorical variables and fed them to a neural network. If this is the case, you can try the following:

  1. add the numerical variables
  2. create and train an autoencoder
  3. use the encoder part to map input to a vector space and perform k-nearest neighbor to it.

You can read the second method described in https://towardsdatascience.com/detecting-credit-card-fraud-with-autoencoders-in-python-98391cace8a3 It uses a dataset with numerical variables only, but since you label encode categorical variables it applies in your case too.

  • $\begingroup$ Thank you. I've tried combining the numerical variables with embedding, but it didn't work well. $\endgroup$
    – Woden
    Oct 7, 2020 at 13:30
  • 1
    $\begingroup$ You are welcome. Have you tried one-hot encoding? I suppose that you have. I guess you have a lot of categories and/or unbalanced data. Maybe you should try something more domain-specific. $\endgroup$ Oct 8, 2020 at 12:46
  • $\begingroup$ My data is kind of sparse, so I encoded it into numbers (in natural order) and concatenate them into one layer. For example, the keys of music are represented by 0-11, and the major & minor are represented by 1, and 0. $\endgroup$
    – Woden
    Oct 8, 2020 at 12:53
  • $\begingroup$ Thank you!! I've got the idea of the autoencoder that it cut down the dimensions for visualization and some other analysis. $\endgroup$
    – Woden
    Oct 11, 2020 at 12:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.