# Knowledge Graph as an input to a neural network

I want to create a neural network that takes as an input a knolwedge subgraph(different types of nodes and different types of edges) to predict some properties. For instance an input in the graph can be :

:Mike :likes :chocolate;
:studies :Biology.



Assuming we have n nodes and r relation types. How can I provide this as an input in a neural network and how will the architecture of the neural network look like.

You can represent this type of knowledge graph as a binary $$n \times r \times n$$ tensor. (You can think of this as a 3D matrix if it helps.)

The first dimension is for the node on the left side of the relationship, the second dimension is for the relation type, and the third dimension is for the node on the right side of the relationship. Then you can represent any relationship between two nodes by inserting ones in the correct indices.

To demonstrate, suppose we have 4 nodes: Mike at index 0, Sully at index 1, chocolate at index 2, and Biology at index 3.

And suppose we have 2 relations: likes (index 0), studies (index 1).

We would first create a $$4 \times 2 \times 4$$ tensor filled with zeros:

import torch

graph = torch.zeros([4, 2, 4], dtype=torch.bool)

""" Output:
tensor([[[False, False, False, False],
[False, False, False, False]],

[[False, False, False, False],
[False, False, False, False]],

[[False, False, False, False],
[False, False, False, False]],

[[False, False, False, False],
[False, False, False, False]]])
"""


To represent the relationship :Mike :likes :chocolate; we would insert a one at the position [0, 0, 2] (0 for Mike, 0 for likes, 2 for chocolate).

To represent :Mike :studies :Biology, we have a one at [0, 1, 3].

# :Mike :likes :chocolate
graph[0, 0, 2] = 1

# :Mike :studies :Biology
graph[0, 1, 3] = 1

""" graph:
tensor([[[False, False,  True, False],
[False, False, False,  True]],

[[False, False, False, False],
[False, False, False, False]],

[[False, False, False, False],
[False, False, False, False]],

[[False, False, False, False],
[False, False, False, False]]])
"""


Of course we can represent any relationship between 2 nodes. Let's add a few more:

# Mike and Sully are friends

# :Sully :likes :Mike
graph[1, 0, 0] = 1

# :Mike :likes :Sully
graph[0, 0, 1] = 1

# I suppose chocolatogoly is a subject in Biology
# :Biology :studies :chocolate
graph[3, 1, 2] = 1


Concerning architecture, there are at least two viable ways to feed this representation to a neural network. You could flatten the whole graph and treat it like a 1-dimensional input. Another option is to retain the 3 dimensions and use convolutional layers (or something else) to extract features (3D convolution works just like 2D convolution).

As for the rest of the architecture, that's up to you!

• @zachdaj Thank you very much for the great answer ! I have two doubts. I tried this approach, but the dimensions are really large and thus my memory cannot handle this. I am aware that this may work on a server. First, is there a way to split it in 2 matrices X and A and then feed both to a NN(for space reason)? Secondly, if I am able to do this on server should I maybe provide an embedding of this matrices? I am not sure whether this should be a separate question. – anascmidt Jul 23 at 7:44
• You may be able to solve your memory problem by using sparse tensors: pytorch.org/docs/stable/sparse.html tensorflow.org/api_docs/python/tf/sparse/SparseTensor – zachdj Jul 23 at 13:35