For a Neural Machine Translation (NMT) task, my input data has relational information. This relation could be modelled using a graphical structure.

So one approach could be to use Graph Neural Network (GNN) and use a Graph2Seq model. But I can't find a good generational model for GNN.

Instead, I want to use Transformer. But then the challenge is how can I embed structural information there? Is there any open source artefact for Relational Transformer that I can use out of the box?

  • $\begingroup$ Can you provide any example of what kind of relational information your input data has? $\endgroup$
    – Valentas
    Commented May 24 at 9:57

1 Answer 1


Some researchers have tried to exploit transformer for graph data. Please read this paper for more details

The implemented code for this has been outsourced on github location with an explanation on youtube video

  • $\begingroup$ do you know is there an implemented version of this? $\endgroup$
    – Exploring
    Commented Apr 6, 2022 at 19:28
  • 1
    $\begingroup$ Hi updated the details, hope it helps $\endgroup$ Commented Apr 6, 2022 at 19:31
  • $\begingroup$ The code does not work :-( $\endgroup$
    – Exploring
    Commented Apr 6, 2022 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.