2
$\begingroup$

I have 100 sequences of the word (i.e., action for completing a task). Each of the sequences contains around 350 actions(115 unique actions but all the actions are not used in each sequence. Some of the actions may repeat). The dataset looks like as below:

Datapoint 1    Datapoint 2 .............  Datapoint 100

Add wall        Add wall                   Add window
Edit wall       Remove Roof                 Add wall
Add wall        Add window                  Edit wall
.......         .........                   .........
........        .........                   .........
Remove door     Add door                    Remove door

My target is to predict the next design actions. However, when I used these sequences in the LSTM model, the prediction accuracy is not so high (35%). For this reason, I am thinking if I can use any embedding model. It is mention-worthy that actions in the sequences are correlated. It means each action has a certain relation with its previous actions and later actions. How can I represent these relationships using embedding? In short, I want to build my own embedding based on the sequence. If anyone help me to provide some reference, paper, it would be highly appreciated.

Improve this question
$\endgroup$
2
  • $\begingroup$ What do you mean by embedding model? Your previous LSTM model must have embed/vectorize the actions somehow too? Can you perhaps add more details about the architecture you tried and the architecture you would like to try? $\endgroup$
    – Jindřich
    Jan 22, 2020 at 9:05
  • $\begingroup$ Sorry for the late reply. In the previous model, I just use one hot vectorization. Now, I am trying to create embedding like word embedding (word2vec, BERT or ELMO). I am thinking if I can use those models to train my own word sequence. $\endgroup$
    – Bloodstone Programmer
    Jan 30, 2020 at 17:16

1 Answer 1

Reset to default
0
$\begingroup$

Almost any embedding method can learn the relationship between those discrete actions.

For example, Python's gensim package has a word2vec implementation that supports training on your own model.

You'll have write your tokenizer that creates pairs of words (e.g., "add wall" or "edit wall") as tokens. Conventional tokenizer will split on whitespace.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.