I have a non-conventional NLP task. I am looking to develop a sequence to a vector model. Instead of employing one-hot encoding for vectorization of the input, I am trying to see if it will be possible to learn to embed for the input text which is usually only one non-human word. For example:

1. GHJJRIDBDL7US = positive
2. LDNF3DM<VAYFI = negative
3. OFNE6NKFE1NNE = positive

ULMFiT is the state of the art now. Any idea if and how it can be applied to this problem? Any idea on how to learn the embedding for such one-word data will be helpful.

  • $\begingroup$ If I understand you correctly, you want to learn an embedding for your input (sequence of words), right?l How about then using Entity Encoding? In Entity Encoding, sequence of words (inputs) can be seen as categories that you want to learn an embedding during a Neural Network training. Google the method, and oyu bunch of tutorials, also I have written a working example in Python for Entity Encoding: github.com/mmortazavi/EntityEmbedding-Working_Example $\endgroup$ – TwinPenguins Jun 11 '19 at 6:13

Browse other questions tagged or ask your own question.